it can beat video games now
>>109721225>our responses API harnessHmmm...
can it beat my meat
>>109721225>wow, we are at 99.9% AGI, that's crazy. So that means cancer, world cancer, inequality, and all other problems will be solved, right? Right?
>>109721225I can’t wait for next year when the models are a hair better and the graph looks exactly the same but the short bar is 99.9% and the long bar is 99.999%
>>109721225That benchmark is shit when you the model are given enough resource it can just copy paste the whole game inside and just search for whatever fits the best.
>video gamesanon...
You mean it can beat Pokemon, a game I beat at 10 years old?
>>109721225Testing with any game that already has been written about (or just is a reskin) in the training data isn't about the ability to 'learn' (which means adapting) but just recalling (which is merely remembering).Mediocre thread, mediocre and clearly overfitted statistic.
>>109721225>it can beat video games nowoh boy I can't wait for the computer to live my life for me
it's not getting through the Safari Zone without help
>we changed the benchmark to make it easier so here is our score with the easier test compared with the old scores on the harder testat this point the continued dumbification of the user base is critical for survival
>>109721225still do not give a fuck about it
>>109721909maybe ask yourself if you're not the dumb one sometimeshttps://x.com/fchollet/status/2095598451115614371
>>109721241>inequality...will be solvedyes, but about 95% of thesubhumans on the planet aren't going to like The Total Solution™
>>109721225>this result is with our special harness, unlike the two previous results in the same graph
>>109721225>the bideo gayme in question
>>109722003
>>109722003>nooo you can't use a harness you have to play videogames with your own hands or it doesn't count
>>109721234c-can I?
Explain to me the difference between the bideo game and the other statistics that barely improved
>>109722103You can use a harness. But don't compare with results without it as it defeats the point of the comparison.
>>109721241It will take time because regulations,but it's already helping. Dont be cynical.
>>109721225If Sol would get 30% with the harness then how much would Opus 5 get with it? 115%?
that means the dataset got mixed in with the training data
>>109722103You will always use some kind of harness, but you should use a normal one.
>>109722103>>nooo you can't use a harness you have to play videogames with your own hands or it doesn't countRedo the old tests with your new harness.
>>109722146You're retarded if you believe it won't replace then erase cattle overtime.Better AI means more useless eaters means more wars and COVID to cull the herd.You're either getting a turbo cancer from an ai-conceived newer illness or dying in the trenches from whatever new risen military conflict.Unless you're not white.
>>109722130yes you can
Literally yesterday some guy posted a graph trying to show ARC-AGI scores plateauing to argue that LLMs were a deadend, lmaoeven the regular harness scores are a substantial improvements:>>109722099
>>109722288you say that, but I don't know, it's so intimidating
>>109721241no goyim will have access to AGI
>>109721225I can't wait to see a surveillance drone that beats the "videogame"
>>109721241They can't even solve the problem of how to get enough power for their datacenters.
Jesus. Fucking. Christ.Welcome, Singularity.
>>109721225Chess Elo?Mahjong Riichi Elo?Dungeon Crawl Stone Soup Elo?
>>109722495>Falling for these overfitted scoresEverything in AI is hype
>>109722146>it's already helpingIs this help in the room with us right now?
this thing is evolving so fast. go back to 2022 and compare with now. it's crazy.
>>109721225It's very impressive how LLM companies still manage to come up with new benchmarks where the new model dominates its older siblings
