a chinese quantized 9B model beats claude and chatgpt
>>108410104Wow, absolute benchmark here.
>>108410104It's well known the 9b models are only 10% weaker than the biggest flagship models like Claude.
>>108410153In (((benchmarks))).In real world shit, the difference is massive.
>>108410160(((Benchmarks))) bad, IQ tests for robots are bad.IQ tests for humans good!
>>108410104Test
penis
>>108410153>107%
>heh, we trained our model on a list of gotchas that people have compiled over the past 2 years and now it doesn't fall for those specific ones>you mad, gaijin?;^)
>>108410616がいじん is a japanese word dipshit
>>108410825nipponese*
>>108410825anon I don't care which made up gender you claim to be, they're all gay
>>108410104can't bump: It was added to the training, all social media "tests" are added.
>>108410104ermmm that doesnt count doe??? its like... the answers are online and stuff... w-what? why dont all the other ones get the answer right since its in their dataset too? uhhhh t-thats racist??? what the fuck? shut the fuck up you piece of shit zipperhead chink?? ughhh i need israeli semen asap i cant take this anymore
>>108410515my iq is also well over 100%.
>>108410189post eyelids
>>108410189You benchmark Musician X.He aces piano, violin, piccolo, and string bass.You put him to play drums.Now, maybe he's good at drums, but also maybe not. I guess amongst human beings he's more likely to play drums well...
>>108410515It's 107% of the performance of the 397B-A22B model. It's plausible because the big model might have a dataset that pollutes the answer. The other factor is that the active parameters on the big model are still smaller (22B) than the "active" parameters of the 27B model.
>>108410153it says there it doesn't even beat the full qwen model, let alone beat an actual good model
>>108410104God I am so tired of these responses. Nigga just respond>You will need to drive the car to a wash anyway.Fucking there. Short and correct.
>>108410189retard, even IQ tests aren't worth shit if the human doing it trained on the test specificaly, even more so if he had the exact answer and can just memorize them.that's what these benchmaxxed models do, they don't get a good answer because of their problem solving capabilities but because they've been trained to remember the answer.
>>108412333you just need to tell it to answer in a short sentence
>>108412433>even IQ tests aren't worth shit if the human doing it trained on the test specificaly, even more so if he had the exact answer and can just memorize them. Don't tell /pol/
>>108413362you think /pol/ is memorizing iq tests?
>>108413390/pol/ says IQ tests are infallible and entire countries and civilizations should be judged based on IQ test results.Studying for an IQ test and memorizing all the answers doesn't matter, IQ tests are still infallible gospels, more religiously important than a Bible.
>>108413406posts like this just make you look retarded
>>108410104Wake me up when it can webcrawl.And no, I don't mean "run out of context on the first page" and I don't mean "slice the page so it no longer can retrieve any data" and I don't mean "use Claude to crawl but end up with just a little better results because he ran out of context before it could get to the answer".For example all of them for some reason have gigantic issues crawling Deadlock's wiki.And why webcrawl? Because it's the only thing that's missing done well to make the local models outright more usable than the online ones like anything Claude does.I tried a few different extensions, spent lots of time trying to get every one of them to work the way it's supposed to and even ended up nuking my local python setup (venv command issues) by accident and restoring it for a few hours afterwards.In the end it sucks regardless of chosen model (believe me, I tried a lot of them and in many different formats), and I'm on a 12GB VRAM card.(GTX4070)
>>108410616>ask an "artificial intelligence" a question a mentally retarded 7 year old niglet would get right>completely fails>ERM, UMM, T-THAT'S.... >THAT'S A CONTRIVED GOTCHA!!!!>THAT'S NOT FAIR!!!!!>FUCKING LUDDITE FAGGOT!!!!!
>>108410160This is the kind of stuff that drives me up a wall. 5% in difference is already a world of difference in knowledge representation, 10% is enough to make the model borderline retarded out of fucking nowhere, and it's not like full models aren't free of their own 'tism either. 35B can be good, but downwards from that is asking to have the model shitting the bed when you least expect it.
>>108413406>/pol/ says IQ tests are infallible and entire countries and civilizations should be judged based on IQ test results.Reddit moment. Now step back for just a second and at least try to be less of a retard just this one time in your retarded life. Is that actually what /pol/ says? Or are they saying that IQ tests are strong indicators of the general intelligence of a particular population? Which they are. It doesn't matter how you feel about it. Stop being a retard.
>>108411119>>108413458the reason we're not winning at AI against Chinese labs with a few million dollars is because we need to give more money to israel. The next time you see a jew, empty your pockets. This is how we win the AI race. More billions to Israel.
>>108410104do this one:Solve a riddle for meA man and a woman are standing at a river. The man has a boat. How can both be brought to the other side?
>>108410104wow they know how to hardcode for gay little riddle benchmark questions. i am super impressed.
