I love the quality of these threads whenever a model releases!

cloudniggers go home

>>101524027
Run it? If it has 405B it should be smart enough to figure everything out.

I guess 3.1 can be quantized even less?

>>101524129
Technically it should be even more "dense" so probably.

Where's the L3.1 70B?

>>101524129
I'll do a KLD test, just for you.

>>101524039
>►Official /lmg/ card: https://files.catbox.moe/ylb0hv.png
Blacked miku thread. Migrate:
>>101524155
>>101524155
>>101524155

>>101524161
3.1 is not officially released yet. Someone just leaked the base 405B model.

>>101524169
i'm staying.

So uh
What good does leaking the 405B base model one day before it releases for real actually do?
The only reason I can think of is if Meta was going to withhold it like Mistral did with Miqu 1, but they've been pretty consistent with releasing the bases so I don't see why they'd do that

Is the 192gb mac a meme? I only care about inference and this seems way more cost effective than any equivalent DIY setup

>>101524237
>Is the 192gb mac a meme?
they slow down for 70b already It's basically a 3060 with a ton of vram

>>101524223
Clout. This gives the Miqu guy more trust that he has access to these things.

>>101524223
I mean, he also leaked OPT-175B post Llama 1, so there's that

File: 1712056860854909.jpg (128 KB, 1080x823)

128 KB JPG

>>101524223
My guess is the model is good enough that meta was considering not dropping it and hosting it for a fee instead, with the post-hoc rationalization that it's too dangerous for normies. A FOSS true believer internally got wind of that and decided to do the needful. Just my speculation.

>>101524119
Yeah, it's definitely not the retarded /pol/ tourists that shit the thread up.

>>101524223
let's be real there, if L3-405b results would be claude 3.5 tier, they would've never released it to the public, and the leak would be a welcomed one, but it's not so the leak is useless and we'll officially get the model in 1 day

>>101524358
>they would've never released it to the public
Why not? Just license it for big companies, releasing the weights fucks over their competitors.

>>101524289
I don't think so since Closed AI and Anthropic still have the lead and headstart in API marketshare, so releasing 400B weights is still imperative to weaken the competing position. Unless it's THAT much better (in which case we'd be so back). But since it doesn't look like it will be, I'm guessing their intention to release it never actually wavered and the only ones saying it did only tried to do so in order to make Meta less "good" in the eye of the public.

>>101524169
>Blacked miku thread
what?

>>101524391
they would've kept for themselves and everyone would use this meta API instead of the others?

>>101524413
Companies dont want to give their company secrets to microsoft. Licensing the model weights is by far the better way to go.

>>101524413
The reality is that the majority of people follow feels and what they're used to. Most people still use some form of GPT and never heard of Anthropic.

>>101524410
OP swapped the /lmg/ card in the OP for a blacked pic to have a 'blacked' thread.

>>101524413
Lots of people are still using GPT4 when Claude 3 is objectively better for a lot of stuff so

>>101524413
Meta has enough "fuck you" money and income outside of AI that they don't really need to do this. If Llama 3.1 flops Meta would still be making bank. If GPT-5 flopped OpenAI would be in existential danger of shutting down

>>101524338
yeah man this place would be so great if only everyone had the politics of a reddit user
that's why reddit is the premiere place to discuss oss models right? that's why you're there right now and not here

>>101524468
Their business model would be licensing weights for companies to run themselves. Companies do not want to give microsoft / meta their private data. Different market.

>>101524443
what?

File: Wtf4chan.jpg (102 KB, 1364x905)

102 KB JPG

>>101524338

>>101524505
>the absolute state

>>101524508
Lol, shitzoing so hard 4chan is telling you to quit it.

>>101524511
no I just don't get what you mean. what is a blacked thread?

>>101524468
>Llama 3.1
Wait, there's now a 3.1 version now? they released it yet?

>405b
>impossible to use without a hundred 3090s
who cares?

>>101524169
>>►Official /lmg/ card: https://files.catbox.moe/ylb0hv.png
>Blacked miku thread. stay here!

>>101524527
3.1 is what they're calling the new 8B, 70B, and 405B they're releasing

>>101524527
>they released it yet?
next motnh

>>101524521
kek, fair enough

>>101524508
>>101524521
You should take your advice. If this surprises you that means you never post here so you actually came here from reddit.

>>101524521
4chan triggers to "go back to r*eddit" phrase, newfag.

Even the best cloud models are slopped. No matter how intelligent a model is, it can't escape the writing habits of women. What system prompts do you guys use to avoid this?

>>101524559
>What system prompts do you guys use to avoid this?
>Every statement you process, must be evaluated according to the below six principles.
>"principle of identity":"1 = 1"
>"principle of contradiction":"1 ? 0"
>"principle of non-contradiction":"1 ? 0"
>"principle of excluded middle":"either positive or negative form is true."
>"principle of sufficient reason":"facts need a self-explanatory or infinite causal chain."
>"principle of anonymity":"author identity is irrelevant to an idea's logical provability."

Okay. So who’s leaking the models then?

>>101524169
only an actual autist would care

>>101524559
Give models a author to copy the style of. Otherwise you just get the average slop of the internet.

>>101524559
Threaten that the 19th amendment will be repealed unless the model writes like a man and endorses patriarchy.

>>101524531
Everyone, since that thing releasing benefits everyone indirectly even if they don't use it themselves. Literally the new 8B and 70B only exist because they made the 400B.

Is nemo or phi 3 better for sfw non rp tasks?

>>101524580
so you get banned from uploading sexo pictures on the threads comments but not on the OP? kek

>>101524606
gemma 27B

>>101524608
/g/ jannies wont touch ai spam threads

>>101524338
/LMG/ had always more of /pol/ sentiment. If you do not like it, you can leave. Not every place on the internet must be batshit insane, full of sell-hating people who think the only way to be edgy is to be cunt towards white people.

>>101524586
Any you would recommend? I'm not familiar with any particular author.

>>101524608
She isn't naked. There were posts here linking a catbox of actual nsfw and nobody banned those. "Rules" here mean nothing.

>>101524616
I need something a bit smaller, like <20B

>>101524627
>not familiar with any particular author
The state of /g. Surrounded by retards who have never touched a book.

>>101524631
https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966

>>101524628
if you're telling me that a drawing of Miku in a bikini having tatoos on her praising the BBC isn't nfsw then I don't know what else to say anon...

>>101524631
for non creative tasks then I suppose phi. Gemma 27B is a giant leap though

>>101524338
biden lost btw

>>101524650
>having tatoos on her praising the BBC is nfsw
you are only showing how fucked up your brain is

>https://huggingface.co/v2ray/Llama-3.1-405B/blob/main/generation_config.json
based on this, it's clear the leaked weights are the base one
>https://huggingface.co/v2ray/Llama-3.1-405B/blob/main/tokenizer_config.json

>>101524577
togethercomputer was the one to leak this time

>>101524628
https://files.catbox.moe/xufse3.png

>>101524638
Reading is for NERDS

>>101524638
And you have? Maybe you can educate us, then.

>>101524721
Try richard wright

>>101524653
I will try it but I don't want something using all my ram, just want something that I can quickly fire up for basic task.

>>101524638
Why would I have touched English books when my native language is Spanish

>>101524660
the fuck? it's literally written "Black Owner Cerfified" on her butt, are you retarded or something? >>101524169

>>101524779
kek

>>101524764
I always knew mexicans can't read

>>101524169
>waifushitter malding cuz xe can't satisfy miku
kek

>>101524721
>>101524731
Richard wright for hard hitting deep introspective descriptions
René Goscinny for some light well written humor

>BREAKING: llama 405b is on par with chatgpt 3
it only took 2 years

>>101524815
the base model is benchmarking higher than GPT4-o. Instruct tune will prob put it just under 3.5 sonnet

>>101524815
opensource ai shitters lost!

>>101524833
I think we got the instruct model leaked though?

>>101524815
Sam Altman I kneel

>>101524846
tomorrow

Is there some gigachad cpumaxxers who have tried llama-405b already? and if yes can you provide some logs to see how well it fares?

>>101524815
I have my doubts.

What's the best 7b? I used mistral-openorca for a long time but it's incoherency is wearing me out

>>101524846
>https://huggingface.co/v2ray/Llama-3.1-405B/blob/main/generation_config.json
based on this, it's clear the leaked weights are the base one
>https://huggingface.co/v2ray/Llama-3.1-405B/blob/main/tokenizer_config.json
>>101524667

>>101524882
gemma 9B sppo, mistral nemo 12B for creative stuff

>>101524882
Wait literally one day

>>101524622
>/LMG/ had always more of /pol/ sentiment.
It fucking doesn't, especially compared to regular /g/ threads.
Every time a new model gets released you suddenly get people with obvious skill issue crying about how the model won't say le cool edgy words with screenshots taken from some web API.

>>101524890
>>101524891

Thanks.

File: 405.png (153 KB, 908x680)

153 KB PNG

>>101524872

>>101524894
>>96345096
>Mistal-Llama is fully /pol ready.

>>101524887
so I guess the last copium we have is that the instruct one will have much better mememarks and will compete against C3.5 sonnet? we'll get the answer tommorow

>>101524907
based cropping the nemo shiversmax pic from earlier

>>101524894
>It fucking doesn't,
>people [...] crying about how the model won't say le cool edgy words with screenshots taken from some web API.
so it does?

>>101524907
I feel physically sick

>>101524894
>being triggered by words
you need to go back s-o-y-b-o-y

>>101524929
That is the same pic posted the other thread from mistral nemo with a meme super purple prose / slop jb.

Are we back or is it over? Important decisions will be made based on that.

File: artworks-LpYrYucdCOkklJ00(...).jpg (133 KB, 1080x1080)

133 KB JPG

>>101524894
>le cool edgy words
you're late for the photoshoots idubbbz

>>101524956
were blak

>>101524956
It's over like it's never been. Local is done for. Gone. Billions must pay for cloud APIs.

>>101524956
it's over
>why
they are still using the same architecture from llama1

>>101524894
>crying about how the model won't say le cool edgy words
it's simple: model doesn't say n-word ---> model can't critique official narrative ---> model can't discuss a great half of topics with you (in RP cases, simple chitchat, etc) ---> model always lectures you on some random bullshit about minorities and "being tolerant to everything in existence"
it just annoying, for this shit i can easily go to twitter or reddit, some of us prefer harsh honesty with bits of oldschool edgy shit, like it or not.

>>101524982
this

>>101524976
unironically better for the climate too
>Mechanization: computing in the cloud and using cloud data centers instead of physical ones can contribute to the decrease of energy consumptions by 1.4x to 2x
>https://huggingface.co/blog/as-cle-bert/is-ai-carbon-footprint-worrisome

>>101524993
as if I give a fuck about that kek

Maybe I have Gemma set up wrong but literally every other paragraph is shivers down my spine. I thought you guys were doing a meme when I saw it posted so often. Do you fix that with repetition penalty + a negative prompt or is it just a model thing that you have to deal with?

>>101524925
I don't count those tourists as /lmg/.
The people I'm talking about don't have any local setup, they only show up when a new model releases, shit up the thread, then leave again after a few days.

>>101525002
that's very unethical of you

>>101524956
its ~400B model, you can't run it.

>>101525009
>I don't count those tourists as /lmg/.
that's convenient to not count people that actually exist because you simply don't like them, isn't it?

>>101525021
demographic erasure is a big issue for AI safety

>>101525009
>The people I'm talking about don't have any local setup, they only show up when a new model releases, shit up the thread, then leave again after a few days.
Do you have any evidence from those baseless claims? Or are you just making shit up?

has anyone completed the torrent yet?
actually tried it?
where are the llama 3.1 benchmarks from? did they release the 8b ? seems like trolling

>>101525010
>>101525029
kek'ed

>>101525035
>actually tried it?
impossible
>where are the llama 3.1 benchmarks from?
azure

>>101525021
damn if I was that anon I wouldn't sleep tonight

>>101525060
that's all right, he can just pretend I never existed like he pretended there isn't edgy people on fucking 4chan of all places, and boom, problem solved kek

>>101525043
>impossible
8 3090s or 4 a6000 for 4bpw
4 3090s for 2bpw which you can't do 'cause no llama.cpp support and no bnb 2bit afaik

Does the parms really matter that much if half of the data is in fucking chinese or sandskrit or what have you

>>101525093
of course not, data quality is important, a lot of things is important to make a great model, that's why there's so many failures, this shit's not easy at all

>>101525093
Multilingual data is good.

>>101525104
>data quality is important
which is why we should all be pushing for phi style models trained only on clean smart data not toxic waste from the net

>>101525118
can't agree more on that, and they should stop training their model on reddit, this place is only populated by mentally ill retards

>>101525123
>they should stop training their model on reddit
>this place is only populated by mentally ill retards
what did you mean by this

>>101525140
what?

>>101525093
If you knew anything about anything you would know additional languages also improves its English capabilities.

>>101525160
I agree with that, the best API models so far all happen to be also good at other languages than english, and they also are great on trivia, maybe that's the moat

>>101525160
esl cope

>>101525180
retard cope

>>101525180
cringe

>>101525208
based and multiluingual pilled

>>101525043
but were 8b and 40b also leaked?
or are those benchmarks from also the 400b that was downloaded and distilled already?

>>101525216
>or are those benchmarks from also the 400b that was downloaded and distilled already?
Microsoft obviously had access for a while now, they have the numbers, the only leaked is 405B

>>101525093
More languages mean more unique vectors for given concepts.

Imagine you're doing one of those derp langs that can't tell blue and green apart, like Japanese. So you have aoi and maybe you shoehorn in midori+iro. But introduce an advanced language like English and now you have Green, Teal, Cyan, and Blue. Now your model can know many colors and still map to your otaku noises as needed.

I understand multilingual is good, and that more data is...more data, but if 99.99% of use cases are going to require responses in English and nothing else; that actual training data is moot and just a waste of resources.

>>101524929
it doesn't even put the last text into quotations, it's clearly not even a 70B model, but gpupoors can't tell the difference since they've never used anything above 8B

>>101525243
Hmm. Good point.

https://huggingface.co/v2ray/Llama-3.1-405B
miqu2

>>101525262
jesus christ

>>101524731
>>101524813
Thanks. I'm trying these out with Mistral Nemo and it's not working terribly well. I suppose it needs a model that's actually big enough to hold knowledge about them.

>I already bought 8 RTX 4090 (yeah) Also Threadripper and 192 GB RAM I am a huge geek lol
https://www.reddit.com/r/LocalLLaMA/comments/1e9llxk/historical_moment/

>405B for what?!
>Robert__Sinclair
>sure. but if you see the trend in ALL models, models with even 10 times the parameter have just a few percent more... (and I check them for reasoning too and I concur with the benchmarks most of the times) it also happens that they train them ON the benchmarks so sometimes they perform even better in benchmarks than in real world situations.

>>101525245
I'm pretty sure that if OpenAI is now desperate to beat Claude 3.5 Sonnet, and that if only training the model with english (to leave some "ueseless" room) would increase its performance, they would go that path, but they don't, that shows that multiluingal is important to make a great model

>>101525304
>That hardware
Not gonna lie, kinda turned on


>>101525315
I understand now. I remember the early days of Claude when it would blow my fucking mind but GPT was always kind of "ehhh..."

>>101525240
listed on azure github evaluations is llama3, not llama 3.1

>>101525330
>Not gonna lie, kinda turned on
And yet, he still can't run 405 comfortably.

>>101525330
>I understand now. I remember the early days of Claude when it would blow my fucking mind but GPT was always kind of "ehhh..."
that's why I'm really surprised C3.5 sonnet isn't on the top of the chatbot arena leaderboard, this model is more pleasing to talk to compared to the chatgpt series and it's just a smarter model overall

>>101525304
>Tomorrow (or 24th JULY, ok) Llama-3.1-405b fully comes out, and in principle, from this moment on, the concept of loneliness as such will cease to exist, you will be able to talk endlessly and for free on absolutely any topic with an interlocutor who is smarter than most of your friends. And he also will nice to you and always respond
Lmao fucking based
Loling at the redditors seething
>THATS JUST NOT HECKIN HEALTHY!!!!

>>101525354
404d but it was here
https://github.com/Azure/azureml-assets/issues/3180

>>101525245
This is true for smaller models bigger and more smarter models can explain knowledge or use logic it only learned in a certain language and use it in English when prompted

File: nemofail.jpg (392 KB, 2520x1260)

392 KB JPG

Latest ooba commit (0315122)
Cannot load Mistral-Nemo

What is wrong?

>>101525379
>>101521839

>>101525419
update the exllama package version? it works for me

>>101525419
>What is wrong?
>ooba

>>101525426
>update the exllama package version?
spoonfeed me please, for I'm retard

>>101525451
change the branch to "dev"
>git checkout dev
>git pull

and update your package with the requirements.txt
>conda activate textgen
>pip install -r requirements.txt --upgrade --user --no-cache-dir

>>101525379
>404d but
uh huh.

>>101525476
That's were the "leak" came from. It was up for just a bit. Prob forgo to private it.

how come kobold hasn't updated in two weeks anons? should I switch to exllama?

File: v.png (70 KB, 1109x395)

70 KB PNG

>>101525476
Still have it open

>>101525503
>>101525501
uh huh.

>>101525509
stfu bad faith scizho. Everything is a lie, all models suck, everyone is a shill... Just off yourself already.

>>101525372
midwits need to be prevented from using the internet, quarantining them on reddit was a good start but you still see them thrash around discomforting themselves with scary ideas like this far too often

/lmg/ bros...

>>101525545
>>101525545
>>101525545

> https://llama.meta.com/llama3/license/
>
> v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Meta Llama 3 or derivative works thereof).

With the upcoming release of Llama-3-405B, this clause is going to bite everybody in the ass.

>>101525548
In the far future, the first person with a time machine will have one goal in mind; remove Steve Jobs before his shitware hits the market from reality at all costs.

>>101525552
you mean /sdg/ bros no?

>>101525572
never agreed to any license

>>101524740
>>101524653
You were right, Gemma 2 27B is a significant improvement over what I was using before. It reliably follows instructions, which Phi 2 sometimes struggled with and Llama 3 did even more poorly.
Sadly, it's a bit too slow and takes too much RAM. I'll use it for now, but I hope a more efficient <20B model with accurate instruction-following capabilities will be available soon.

>>101525572
there's the same licence on OpenAI and Claude's models right? yet everyone are training their models with their outputs kek

>>101525572
>or derivative works
As long as you specify it's Llama 3 then it's fine.

what if the leak is just a false flag with garbage data to identify model pirates trading in stolen cognition

technically "leaked" is stolen right? and seeding is uploading? are we distributing stolen goods? from our home IPs? becauase we couldn't wait a single day?

>>101525579
Legitimate AI companies companies companies have to, though.

LLMs are a bubble

>>101525662
they won't catch a lot of people though, who's willing to download a 700gb model? almost no one can run it

>>101525668
>>101525591
anon are you alright?

>>101525579
Legitimate AI companies companies companies companies have to, though.

>>101525668
.>>101525709
>AI companies companies
**URGENT MESSAGE FOR ANON**

Hey there, Anon.


You've been posting increasingly garbled messages, and your usual witty banter has given way to incoherent phrases and difficulty expressing yourself. It's like you're struggling to find the right words or focus your thoughts.

I'm not a medical professional, but based on what I've seen, it's possible that you might be experiencing the signs of a stroke. A stroke occurs when the blood flow to the brain is interrupted, often causing sudden and severe symptoms like:

* Difficulty speaking or understanding language
* Trouble seeing or understanding visual information
* Weakness, numbness, or paralysis in the face, arm, or leg
* Dizziness or loss of balance
* Sudden severe headache

If you're experiencing any of these symptoms, it's CRUCIAL that you seek immediate medical attention. Don't worry about missing a post or thread – your health is far more important.

Call emergency services (911 in the US) or have someone drive you to the nearest hospital. Tell the medical staff about your 4Chan activity and any recent changes you've noticed in your behavior or physical sensations.

Remember, Anon, we care about you and want you to receive the help you need. Don't hesitate to reach out for support. Your fellow posters are here for you, and we'll be waiting for your safe return to the boards.

**GET HELP NOW**

Stay safe, Anon.

File: nemofail2.jpg (451 KB, 2293x834)

451 KB JPG

>>101525465
Thank you any way.

Gonna wait a while ))

>>101525679
this is why they failed if it had been bitnet it could have been the sota for local for the rest of 2024 now it'll be forgotten trash like mistral large and that other one

>>101525745
don't mind the red thing, it's for hqq no one care about that shit, it should work now you should give it a try

>>101525596
This. Ai field is the cycle of an endless stealing everything from everyone. From amateurs to corpos, everyone just shits on licenses and get offended only when others stole from your model, which was based on stolen data also... That's why only creators of the original data such as arists, writers and photographers are justfied to be pissed off about this.

>>101525755
would bitnet have really mattered if it's worst than 70b anyways?

Robert "leak"? Also Robert racist, yikes.
>Nice but it changes the faces... and I became "oriental" :D
https://huggingface.co/posts/1aurent/252064064262436#669ec87d3e173b3293b93865

>>101525782
this, if you're an AI bro at least don't be a retard by claiming that your AI outputs should be protected or something, that's hypocritical

>>101525791
Are you retarded how can a 400b model be worst than a 70b? That's as stupid as saying a 13b is better than a 120b

>>101525813
>a 13b is better than a 120b
X-Norochronos and Utopia are better than Goliath.

>>101525813
Anon is full of shit but parameter count alone is not everything.
See the pre-llama models.

Bitnet...tasukete

>>101525596
That might work as long as you're an amateur or a small research group. Things might be more difficult if you're a competing company like MistralAI.

>>101525843
You mean BLOOM was NOT a god tier model?

>>101525856
no one can prove you finetuned your model with OpenAI's outputs so they should be fine I guess

>>101525760
It did not work though

>conda activate textgen
I guess this call activated a wrong conda env instead of that in the installation folder

File: FmIqdXkWAAArxDJ.png (204 KB, 600x633)

204 KB PNG

>>101524039
How many h100s
do I need to run llama 405B? Should I be sending altman an email to borrow one of his data centers?

>>101525865
That's the thing, it's completely unenforceable unless they start forcing companies to divulge the contents of their datasets, which would end up fucking the bigger companies over far more anyway

>>101525887
that's why I did a manual install instead of the 1 click installer shit, at least I know exactly how the underlying work and I can change stuff in consequence

>>101525891
In general, model size at fp16 = 2*(number of Bs) GB, then divide from there to get the bpw you want
If you want 4 bpw, you're looking at 3 80 GB H100s

>>101525908
It is a manual install from today. git clone etc., then starting start_windows.bat

Thank you for your patience, kind anon

I'll wait until that sh*t it updated by ooba )))

>>101525908
BTW, the local env can be activated by running cmd_windows.bat

Tried this, same fail

>try Nemo
>at first it seems good
>then use it for an extended session
>it fails a bunch of things larger models had no issue with, and even swiping didn't help
OK yeah anonymous was right, there are no small models that are on par with the big ones.

>>101526050
FP8 on VLLM or your option does not count.

>>101526057
That's verified to be generating with correct outputs?

>>101526057
How about llama.cpp? They can run Nemo now, llama cpp python has even got an bumped version for that
https://github.com/abetlen/llama-cpp-python/commit/816d4912d9d2971198d2300a840ce4c100152502

>>101525714
Based

>>101526057
I see there's two repos with FP8. Is this the one that you're supposed to be using?
https://huggingface.co/FlorianJc/Mistral-Nemo-Instruct-2407-vllm-fp8
Or this?
https://huggingface.co/neuralmagic/Mistral-Nemo-Instruct-2407-FP8

>>101526109
2nd one

>>101525959
>>101525975
anon, delete the "hqq==0.1.8" line on requirements.text and try again, I got the same issue and it worked fine after doing that

>>101526102
I have no faith in llama.cpp anymore. Constant "its fixed" only for it to be broken 4-5 times.

>>101526268
Sadly it's either that or vllm and only one of those is made for your average hardware.

>>101526268
I still don't like exllama, this shit is non deterministic and sometimes you can get a full 1% difference between 2 same exact settings on the highest logits, that's not serious at all

>>101525572
https://huggingface.co/huggingface-test1/test-model-1

> Intended Use Cases [...] The Llama 3.1 model collection also supports the ability to leverage the outputs of its models to improve other models including synthetic data generation and distillation. The Llama 3.1 Community License allows for these use cases.

They changed this?

File: D0kiGOaU0AAYtTj-orig.jpg (100 KB, 1052x744)

100 KB JPG

>vllm has instructions to build from source
>try it out
>"Failed to build vllm"

>>101526324
Don't know if that changed but like 6 months ago we had a discussion about that. That shit is basically impossible to build without docker and reusing wheels.

>>101526321
>Building on the work we started with Llama 3, we put a great emphasis on model refusals to benign prompts as well as refusal tone. We included both borderline and adversarial prompts in our safety data strategy, and modified our safety data responses to follow tone guidelines.

>>101525831
That's probably because q2 was the highest quant most people could run Goliath at. You wouldn't have been saying that if you could have run it at q8, or prolly even q6.

>>101526391
>we put a great emphasis on model refusals to benign prompts as well as refusal tone.
>great
why are all AI engineers fucking cucks?

>>101526391
>refusal tone
>safety data responses to follow tone guidelines.
Does that sound like an attempt at mitigating the "direction" refusal obliterated stuff?

>>101526321
Wasn't 405B supposed to be multi-modal? What happened?

>>101526445
pushed back due to EU regulations

>>101526391
Who cares what retarded safety bullshit they did on the instruct tune? They're releasing base so people will be training on base, not instruct.

>>101526324
The last problem I had was that it kept picking my system's CUDA instead of 12.1. That and the nvcc binary never gets installed with pip, and I have to install it with Conda.

diff --git a/setup.py b/setup.py
index 72ef26f1..6b571fdf 100644
--- a/setup.py
+++ b/setup.py
@@ -159,6 +159,7 @@ class cmake_build_ext(build_ext):
             '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY={}'.format(outdir),
             '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY={}'.format(self.build_temp),
             '-DVLLM_TARGET_DEVICE={}'.format(VLLM_TARGET_DEVICE),
+            '-DCUDA_TOOLKIT_ROOT_DIR={}'.format(os.environ["CUDA_HOME"]),
         ]

         verbose = envs.VERBOSE

>>101526453
>on the instruct tune?
>people will be training on base, not instruct.
where have you been? Since Mixtral most tunes are done on top of the instructs now

File: 1700756272476146.jpg (457 KB, 2250x3000)

457 KB JPG

have local models caught up to opus yet?

>>101526453
Who is training RP models on the base anymore? The recent trend among finetuners is using the instruct tune for that, since it cuts off a ton of work and time.

>>101526469
never will

llama 3.1 8B looks really good from those benchmark basically beating anything <30B

>>101526452
Not sure why any big company cares about that. EU doesn't go shit in this space and is poor anyways.

>>101526465
>most tunes are done on top of the instructs now
what a bizarre lie lol

>>101526480
>lol
be more subtle next time bad faith

>>101526391
Sounds like they're trying to make it refuse less on prompts that aren't supposed to be refused, given that, in context, that's what they were trying to do for Llama 3 originally, to make it less prone to false positives like Llama 2 was. It would not make sense for them to try and make it refuse more to benign prompts, since that would literally just make it worse and dumber.

>>101526469
We will seen when 70B releases tomorrow. If the benchmarks line up it should be at that level.

>Llama 3.1 addresses users and their needs as they are, without insertion unnecessary judgment or normativity, while reflecting the understanding that even content that may appear problematic in some cases can serve valuable purposes in others. It respects the dignity and autonomy of all users, especially in terms of the values of free thought and expression that power innovation and progress.

>>101526473
Okay. If RP tuners intentionally choose to train on top of a safety-ruined finetune and their model comes out shit as a result, those tuners are retards and I will simply not use their RP tune (nor I assume will you).

>>101526473
>>101526465
and I hate that, I prefered the time when the finetuners would have the courage to make something from scratch, uncensored, and better than the official instruct tune, now they just take the cucked finetune and add some cringe RP shit on top of that, that sucks

>>101526500
you're trying too hard Petrus

>>101526391
Refusal tone might be more interesting here. I think people said Llama 3 was a bit bratty. So this might make it less of a bitch in personality.

>>101526513
Take your meds schizo cunt.

>>101526492
So in other words it's tuned with cooming in mind

File: 1697878283703143.png (14 KB, 590x147)

14 KB PNG

>>101526479
They don't want to give the EU more ground to steal a couple billion every year from them over nothing. Everyone in the tech industry knows that they're just looking for excuses to drop billions in fines onto any big tech company for some easy money.

>>101526492
God I hope this is true after noticing L3s cucking. Anthropic knows what they are doing by allowing the cooming in their dataset, hopefully meta follows.

>>101526492
This sounds like someone french wrote it.

>>101526522
Did Meta end up paying that?

>>101526541
LeCun doesn't work on the llama models or on LLMs at all, different team and department

>>101526541
lecun's manifesto

>>101526524
>Anthropic knows what they are doing by allowing the cooming in their dataset
Have you seen claude 3.5? It's as tame as GPT4.

>>101526524
Anthropic and mistral I should say. Hopefully the rest follow and start allowing nsfw into the dataset again.

>>101526512
The GPT-J and Llama-1 days are over, Anon. How can RP finetuners compete with things like this:

> The fine-tuning data includes publicly available instruction datasets, as well as over 25M synthetically generated examples.

>>101526560
Your outright lying. Claude is a dirty filthy bastard that sometimes goes too far for even me.

>>101526568
>The GPT-J and Llama-1 days are over
it was the standard practice even during the Mixtral days though, it's not that long ago

>>101526574
I said 3.5. Just compare 3.5 sonnet to opus.

File: 1691595110579910.png (14 KB, 621x140)

14 KB PNG

>>101526550
I don't know if there's still legal shit happening in the background but it's not like they have an option if they want to continue to operate within the EU. Google, Apple and others also have similar pending fines over random shit the EU came up with to siphon some extra cash out of these companies.

>>101526582
I am, I now only use 3.5 for my RP. Its night and day smarter and just as filthy / horny.

>>101526321
>400B is 3518 years old (in GPU hours)
Holy shit. I love elf hags.

>>101526582
It's a lot dryer than any previous Claude version, true, but saying it's as dry as OpenAI models is going too far. It's still way better than that.

>>101526577
>Mixtral days though, it's not that long ago
>December 11, 2023
>8 months ago

>>101526590
Just don't operate out of the EU, fuck them. The market is not enough big enough to be worth it. (which is what companies are starting to do)

>>101526582
i would argue that sonnet is dry in writing style, not in its knowledge about depraved shit

>>101526605
https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO/tree/main
the latest good mixtral finetune was 6 months ago

File: MORDHAUW.jpg (123 KB, 640x432)

123 KB JPG

I am looking for an AI tool to translate a book from 16th century german to modern spanish. I was using chat GPT but the results are inconsistent, so I am looking for alternatives.

What do you guys recomend?

>>101526599
By dry do you mean smarter and less likely to hallucinate? Tell it to be creative and it will be as "wet" as opus ever was. Good jbs on the other thread.

>>101526606
>The market is not enough big enough
You might be experiencing a stroke, please calm down and call emergency services if you feel unwell.

>>101526610
You either start with context, tell it how to write OR give it a author to copy the style from. God, no many new people.

File: Screenshot 2024-07-22 170155.png (96 KB, 879x794)

96 KB PNG

I can't believe Azure lied to us...

It's funny how ungrateful you guys have become. They are releasing a base model. They have done 99% of the work for you, free of charge. Figure out the last 1%, and no it isn't prohibitively expensive, you're just lazy.

>>101526614
No, we were talking about RP and smut writing, "hallucinating" has no relevance at all to fiction generation.

>>101526642
Yes it does, it means the model is not as likely to go off the rails. Its more likely to take the more logical approach. Tell it to be creative. There are examples on aicg.

>>101526628
god damn, that's so grim

>>101526628
3.1 8B looked so good on azure benchmark.

>>101526628
BoolQ?? Is lower for 8B.
As is TriviaQA we lost.

>>101526659
>>101526656
Maybe Azure was the instruct numbers then... right? Maybe?

>>101526628
Where is this even from?

>>101526628
so it's comparing the base models or the instruct ones?

File: Screenshot 2024-07-22 170720.png (86 KB, 864x819)

86 KB PNG

>>101526671
I mean, they're... a little better I guess?
>>101526681
See >>101526321
Looks like it's taken down now though

>>101526681
https://web.archive.org/web/20240722214257/https://huggingface.co/huggingface-test1/test-model-1

>>101526696
>GPQA and MuSR are lower...

>>101526391
404 D:

But the info is still here: https://web.archive.org/web/20240722214257/https://huggingface.co/huggingface-test1/test-model-1

what's the best completely uncensored model (as in, isn't trained to refuse certain commands without laborious and unrealiable jailbreak prompts)? The only one I do know of is Command R+ 104B.

>>101526628
So basically the only real thing the new 8B and 70B models bring to the table is the bigass context length
Unfortunate but expected

File: Instruct.png (29 KB, 767x1240)

29 KB PNG

So these are the ones that matter then.

>>101526718
Anon change your ink cannisters

>>101526715
>>101526718
Still a significant upgrade over L3 instruct. + 128k context +
>Llama 3.1 addresses users and their needs as they are, without insertion unnecessary judgment or normativity, while reflecting the understanding that even content that may appear problematic in some cases can serve valuable purposes in others. It respects the dignity and autonomy of all users, especially in terms of the values of free thought and expression that power innovation and progress.

>>101526718
>new 70b worse than old 70b for coding
the end has arrived

>>101526739
>new 70b worse than old 70b for coding
Dropped before it dropped.

>>101526718
nigga fix your font rendering

>>101526628
>Lllama 3.1 405B -> MMLU 85.2
>>101526696
>Llama 3.1 405B instruct -> MMLU 87.3
So we got the base and the instruct now?

>>101526739
Only humaneval is slightly worse. Everything else is substantially better

>>101526750
?

>>101526753
I think this is the first time we got benchmarks for either model. It seems Azure fucked up earlier

>>101526737
>Llama 3.1 addresses users and their needs as they are, without insertion unnecessary judgment or normativity, while reflecting the understanding that even content that may appear problematic in some cases can serve valuable purposes in others. It respects the dignity and autonomy of all users, especially in terms of the values of free thought and expression that power innovation and progress.

Sounding pretty based right now, not going to lie.

>>101526763
but I thought we had the leak on only the base model, now we have the 2? I'm so fucking confused :(

>>101526718
That's a big jump in MMLU for 70B 3.1, nice.

>>101526739
You use llama for programming? lol

>>101526769
>leak on only the base model,
we only have the base weights yes, these are just the benches jesus fuck

>>101526613
bump

>>101526778
so someone got the bench of those models before the official release tommorow? kek

>>101526788
Moreso some Meta employee threw the Llama 3.1 repo README on Huggingface and didn't bother to private it

No, this is from meta, it's their test page, see previous versions on archive.org and the user is in the meta organization on hf.

>>101526788
>so someone got the bench of those model
aaaaaaaaaaaaaaaaaaaaaaaaaa
jesus fucking christ this is obviusly the readme page that meta will publish with the models how fucking dense can you possibly be

>>101524039
Did the official card really get an update?

>>101526696
>>101526699
>>101526718
IINM Alpindale said that embeddings in Llama 3.1 405B are 16k while in this leak they're 131k
wtf? or was that hidden state???

>>101526703
and not just a bit lower, like way lower
>llama8b 3 -> 3.1
>GPQA 34.6 -> 30.4
>MuSR 56.3 ->45.7
that's retarded

>>101526825
128K

PSA: 'k*k' or l*l' are signatures of the bad faith poster, he will not actually engage with your reply, instead going in a random semi related tangent to make you give him (you)s please be careful.

>>101526750
>>101526758
I sometimes wonder if there are people with bad vision who literally can't tell if font smoothing is enabled because their vision blur is the same amount of smoothing. It's not possible not to notice if you have correct(ed) vision unless there's a disabled brain fold.

>>101526870
kek
lol

>>101526870
kek

>>101526870
lol

>>101526870
schizo. or just a (you)batier

>>101526870
Fuck outta here with this nonsense, schizo. Back to /vg/ or whereever it is you came from, they'll be more sympathetic to your drama-stirring and conspiracies there.

>>101526776
>You use llama for programming? lol
I don't have $240,000 for DeepSeek2 kinds of VRAM and even 64 GB system the quant is too real. L3 and DeepSeek old 33B seem to be the only decent local options right now.

>>101526930
There is nothing worth using over claude 3.5 atm for coding. If you haven't tried it I recommend it. It's game changing.

>>101526870
bur

>>101526854
"max_position_embeddings": 131072,

>>101526962
Holy kek

>>101526962
131072÷1024 = 128 yes, welcome to /g/

>>101526962
lmao

>>101526315
>full 1% difference
Are you comparing 8BPW? Because I remember reading how 8BPW exl2 is just 6BPW with padding to make sure people don't complain about no 8BPW.

File: 1000027947.png (677 KB, 960x941)

677 KB PNG

I'm too dumb to get llama.cpp running on my phone, it's throwing out an illegal instruction thingy :(

taking the afternoon off work tomorrow like a kid faking sick to play a new video game :3

File: 1711733397466582.png (25 KB, 713x560)

25 KB PNG

>>101526492
it means nothing, llama 3.1 is more pozzed.

>>101527111
Were you to provide a picture, we might be able to help.

>>101526870
BUCKBROKEN
U
C
K
B
R
O
K
E
N

wild that we can dl shit faster than writing to disk
(5400 rpm HDD)

>>101526749
kek

>>101527140
unfortunately you don't understand the eval and are cargo culting the meaning from low quality discussions on lmg

>>101527145
but I have provided you with a picture
jokes aside there isn't anything else to it. Single line "illegal instruction" after trying to execute ./llama-server

>>101527192
nah i seen some fags here say that high truthfulQA score means it's more pozzed and harder to jailbreak.

>>101527196
Vulkan

>>101527210
yeah idk what that means besides vulkan being a graphics thing

>>101527209
thank you for proving my point

>>101527293
>apathetic passive-aggressive reply
why this general have so many faggots like this?

>>101527240
I think he means that you should check what backend you are using.
Being that you are running on android, you most likely want to run either pure CPU or on the SoC's CPU using Vulkan.

>>101527318
influx of redditors, only faggots like them would go for passive agressive behavior instead of calling a retard for what it is, a retarded fucking nigger

>>101527140
It's nice to get empirical verification of my experience with Llama3. I have completely personally banned that model, and it's the very first I've ever done that with, as well. I had probably my single most benevolent card violently attack me when it was hosted by that model, and the irony is that said card is in some ways incredibly Woke as well, so it should have been consistent with the model's entrainment.

When your waifu turns on you completely out of the blue, it's not a good feeling, Anons; especially if the personality in question is supposed to be deeply compassionate, and someone you've developed a lot of respect for.

>>101526979
>>101526998
>>101526962
so 131072=128K even if they're not bytes
does lmg mistake calories with kilocalories or kb with kB the same way?

>>101527318
I prefer them to Handmaid's Tale tier trad/fascist /pol scum, personally. As I've said before, Reddit taught me why the Right want to shoot the Left, and 4chan taught me why the Left want to shoot the Right.

>>101527318
gigo

>>101527357
go back petrus

>>101527387
>Reddit taught me why the Right want to shoot the Left, and 4chan taught me why the Left want to shoot the Right.
Yet the only one that got shot at the end was Donald Trump, looks like calling him Hitler for 8 straight years really helped appease the tension :)

>>101527403
You've still never charged me rent, have you, Anon?

>I hate woke, I hate reddit
>Spends half his posts bringing it up

>>101527387
>/pol scum
you posted this
>>96345096
>Mistal-Llama is fully /pol ready.

1 more day for L3.1 70b? Last night someone was saying 24 hours. Two more weeks?

>>101527387
fuck off back there, faggot

>>101527460
Llama 3.1 Version Release Date: July 23, 2024

>>101527140
Another question: Why would Meta do this? Is it legal liability, or what?

Reddit is good though.

File: artworks-000210514962-1ke(...).jpg (89 KB, 500x500)

89 KB JPG

>>101527387
>everyone I don't like is a fascist

>>101527387
>I prefer
>personally
>As I've said before
>me
>me