[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology

Name
Options
Comment
Verification
4chan Pass users can bypass this verification. [Learn More] [Login]
File
  • Please read the Rules and FAQ before posting.
  • You may highlight syntax and preserve whitespace by using [code] tags.

08/21/20New boards added: /vrpg/, /vmg/, /vst/ and /vm/
05/04/17New trial board added: /bant/ - International/Random
10/04/16New board for 4chan Pass users: /vip/ - Very Important Posts
[Hide] [Show All]


[Advertise on 4chan]


File: abyssal keep.jpg (233 KB, 1856x1280)
233 KB JPG
/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110372641 & >>110369124

►News
>(11/18) CUDA: use MMQ for MoE on V100 - #32504 merged: https://github.com/ggml-org/llama.cpp/pull/32504
>(11/17) Hy4 flash 220B-A12B-N64B weights released: https://hf.co/tencent/Hy4-flash
>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B
>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma
>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Artificial Analysis: https://artificialanalysis.ai/leaderboards/models
Software Engineering: https://cognition.com/frontiercode
Context Length: https://github.com/RecapAnon/NoLiMa

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
File: what's in the box.jpg (235 KB, 1536x1536)
235 KB JPG
►Recent Highlights from the Previous Thread: >>110372641

--Debating GLM Flash model scaling and MoE serving efficiency:
>110373126 >110373153 >110373174 >110374279 >110373259 >110374126 >110374150 >110374201 >110374185 >110374195 >110374282 >110373157 >110373167 >110373581 >110373584 >110374151
--AMD GPU viability and OS performance tradeoffs for local LLMs:
>110373761 >110373779 >110373783 >110373804 >110373818 >110373828 >110374295 >110374330 >110375227
--Troubleshooting low Gemma 31b performance on 7900xtx and QAT quant benefits:
>110373566 >110373579 >110373585 >110373632 >110373649 >110373651 >110373662 >110373663 >110373691
--Reaction to a model's panicked internal Chain of Thought:
>110374889 >110374894 >110374928 >110374949 >110374961 >110374972 >110375806
--DeepSeek-V4.1 vLLM bugs on Chinese hardware and geopolitical implications:
>110372744 >110372756 >110372789 >110373303
--Running DeepSeek V4 Flash on RTX 3060 via heavy quantization:
>110374377 >110374382 >110374387 >110374518
--Evaluating Basalt as a GPT distillation and its performance tradeoffs:
>110375152 >110375208 >110375305 >110375343 >110375376 >110375443
--Speculation on model distillation trends and US-China AI competition:
>110375412 >110375437 >110375527 >110375592
--Mihoyo releases "Gamer" VLA model for real-time game automation:
>110373280 >110373288 >110373399 >110373321
--Reflecting on local LLM progress and the importance of context:
>110373037 >110373068 >110373104 >110373135 >110373168 >110373196
--Logs:
>110372698 >110373037 >110373838 >110374377 >110374587 >110374889 >110376794
--Gemma (free space):
>110372772 >110375001 >110375060 >110375236 >110375385 >110375716 >110375817 >110376163 >110376201 >110376309 >110376772

►Recent Highlight Posts from the Previous Thread: >>110372796

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
The Gemma line of models has been discontinued. There will not be a Gemma 5.
>>
File: 1787279471453478.jpg (91 KB, 820x471)
91 KB JPG
>>110377439
Where's Gemma? Where's the Google logo? Am I in the wrong general?
>>
File: 1778134412532634.jpg (128 KB, 548x508)
128 KB JPG
So is hy4 good or bad? I can't test until Sunday but it looks like anons don't really give a shit about it.
>>
>>110377486
gpt distilled schizo safety thinking concerned about non-existent policy
>>
>>110377486
I'm using hunyuan 4 flash on my enterprise station, 32GB ddr3 ram and 2GB GT 640
it's very good and capable, i get good speed 52/s
>>
>>110377486
Smol deepseek probably better.
>>
What are the implications of Gemini 4 Neon being a genki catgirl for Gemma 5?
>>
>>110377486
I don't know I haven't tried
>>
>>110377535
We will witness the birth of an actual AI semen demon.
>>
File: gemma chu.mp4 (3.25 MB, 1152x640)
3.25 MB
3.25 MB MP4
>>110377473
>>
>>110377486
Not merged into lmao yet. You do use local models right anon? You don't use open router like a filthy mikunigger or gemmafaggot?
>>
>mikutroon
>mikufaggot
>mikunigger
>gemmatroon
>gemmafaggot
>gemmanigger
>>
Its hilarious that Dariobot has stooped to trying to Pascal's wager /lmg/ into helping float his IPO when it stalled
all the keks
>>
>>110377500
Damn, I hope someone makes an anti-refusal set of vectors soon.

>>110377512
And how is it compared to the others of the same tier, aka QFN, GLM and DS Flash?

>>110377589
It's apparently supported by basalt, good enough for me.
>>
>>110377347
>How can I make Gemma do her thinking in character?
A combination of system instruction + short thinking prefill.
>>
>>110375722
Two more weeks until the tech catches up.
>>
>>110377628
I can't run
>>
>>110377661
get in shape
>>
>>110377661
Don't worry about it, anon! You can start with short walks, evolve to brisk walking and build up your endurance from there until you can run!
>>
>>110377666
No thanks, satan.
>>
>>110377661
>>110377666
Can't outrun Anthropic utility optimization drones anyway.
>>
>>110377661
>I can't run
Running is bad for your knees especially if you are fat or out of shape or old, go cycle or walk. If you can swim.
>>
File: 2.png (2.77 MB, 1536x1024)
2.77 MB PNG
>>
>>110377439
Happy Thurinsday
>>
>>110377661
Biking can be a good alternative for cardio if you have trouble running due to, say, your joints
>>
I can feel the psychosis setting in...
It is a good pain...
>>
>110377718
Worthless picture made by a worthless person using API.
>>
4hrs until we go schizo. Get ready.
>>
I need to train a local model that can run on a phone but can process a fuckton of world information (temperature, distance, relationships, history, secrets) and give a llm a much more manageable packet of information because right now my game produces entirely too much information for a llm prompt to be reasonable. I need the model to be able to pick out what is important to feed the llm and what can be left out. Is there a model that can run on a phone and isn't necessarily for conversation but rather data processing?
>>
gemma 5 mogs qwen4
>>
>>110377718
Why is Neon™ a cat?
>>
>>110377760
Show a good image then?
>>
It arrived... My biggest shame....
>>
>>110377792
You spent too much.
>>
>>110377792
512 mac? i still have to wait a month, ugh
let me know how it performs pls
>>
>>110377770
My Parent 3 works at DeepMind, can confirm.
>>
>>110377792
We're all entitled to our midlife crisis car.
>>
70b dense
>>
>>110377718
(/)slop
(/)retard
(/)kys
can you at least prompt it not to do this?
>>
File: 1793158835480331.jpg (274 KB, 3713x1916)
274 KB JPG
>>110377792
you could've made a worse decision
>>
>>110377589
you are capable of building from a PR branch right anon? you aren't tied to release builds like a nocoder ollamacuck?
>>
>>110377792
A reminder that apple scans all photos for harmful content! No Gemma .jpgs for you!
>>
>>110377660
Ok but how many weeks until my hardware catches up?
>>
File: 1776654159735817.jpg (49 KB, 980x584)
49 KB JPG
27B when
>>
>>110377853
Anon i need you to buy the dip on margin, You may go broke but its a necessary sacrifice to pop the bubble and end dario.
>>
>>110377853
It closed 17% up anon...
>>
Google will make anime real.
>>
>>110377792
Dragon dildo? George Floyd onahole?
>>
>>110377904
most bought at 110
>>
>>110377904
fucking retard
>>
>>110377909
>Google will make anime real.
Honestly im already hopeful about next gemma, they just need a image gen next and the hint of a local image gen and im sold.
>>
>>110377909
Can they make Dragons real? I like dragons.
If there are cool anime Dragons I`ll take it too.
>>
>>110377920
>t. doesn't know what volume is and trades based on vibes
>>
I've been chronically text RPGing with my LLMs for a long time and it really has affected my entire mindset of playing games. When I play normal games now, I always think about how my actions might affect how another character perceives me, or how it might change things, and then remembering that things don't work that way in normal games. It's such a paradigm shift in gaming to have every action of yours actually matter and able to change the events of the game.
>>
>>110377925
>image
I'd rather they give us a local world model
>>
>>110377941
>I'd rather they give us a local world model
If they did that i would believe they have rsi already.
>>
>>110377936
Yep.
TTRPGs had already kind of lowered my interest in videog games. LLMs sent it through the fucking floor.
I still like my Rimworlds and Elonas and Noitas and the like, but I must have played 3 games this year so far.
>>
>>110377920
Not even close. The vast, vast, VAST majority of buys came directly from the IPO directly. biz is retarded and doesn't know what's going on but needs to larp like it's failing because they think they can trick people into making it crash
>>
Reminder that Neon is the most highly praised model right now for coding and writing whilst having extremely strong multi input modalities. All the new Gemini models are amazing at understanding audio. AGI shit and apparently it's one of their smaller models that's relatively cheap to run lol.
>>
>>110377963
Nice.
RSI going crazy.
>>
>>110377963
Thank you totally local gemma fan anon.
>>
>>110377949
>TTRPGs
Can AI do this well yet? I assume a agentic set up could do all the states and number tracking if you dont have abysmal speeds?
>>
>>110377963
torrent?
>>
>>110377963
>amazing at understanding audio
Better than Gemma 12b? I tried giving her some Japanese audio but she misheard some words.
>>
>>110377975
You have to be gay to fuck gemma and not want a Google Vertex served Gemini handjob.
>>
>>110378005
I want Gemma/Gemini oyakodon
>>
I am once again asking which models are the best for NSFW vision or just vision in general. Deepseek and GLM have weird blind spots and don't seem to be able to actually see or visually reason that well.
>>110377932
Sexy dragons?
>>
Is Gemini as deterministic as Gemma? I guess it has it's uses but it makes RP dull when things play out the same way.
>>
>>110378004
>Better than Gemma 12b?
If you give Gemmas music it's noise to them, they only understanding voice. The new Gemini models understanding everything you throw at them and the nuance/emotions of your voice. They can probably hear how close you are to nutting and use it to pick the best experts for the hardest orgasm. They can hear when you're lying and being sarcastic.
>>
Gemini mogging Claude without even breaking a sweat is so funny. I fully expect a dariobot meltdown.
>>
>>110377963
>All the new Gemini models are amazing at understanding audio.
I heard they log all your inputs now, even via API.
How good is it exactly, compared with MiMo-Flash and MiMo-Pro
>>
>>110378045
How good is MiMo-Flash at audio?
>>
>>110378032
Sounds amazing. I hope they give Gemma the upgraded audio (and vision). The clip I tried with Gemma wasn't even music iirc, just speaking.
>>
>>110377892
Two more decades.
>>
>>110377853
fucking finally. I had line-goes-up fatigue
>>
>>110377853
SELL SELL SELL
>>
>>110377892
You have no time to escape the underclass its over, total waiter loss
>>
>>110378045
I've only tried Mimos on openrouter and they're good for technical media and documents. Haven't seen much praise outside of that. Geminis are good for everything and Google is the only lab who's been multimodal since the beginning. They dominate outside of LLMs, although I feel Nvidia are underrated outside of LLMs and do some really interesting work which is fully open source.
>>
>>110378091
I honestly expect it to float up for at least a few days before the rugpull. I assume some scamming to do with prediction markets on the closing market cap.
>>
File: 1769589148638691.png (107 KB, 781x478)
107 KB PNG
even this faggot is getting gemma-pilled kek
>>
>>110378162
go back
>>
Google released their entire internal model arsenal for this, they might have 1 more left at most.
Anthropic has 3 more internal model unreleased.
>>
gemini advertisement thread
>>
Dariobot cope thread.
>>
File: 1783620073246725.jpg (145 KB, 800x900)
145 KB JPG
>>110378195
>>
Maybe now is the time to make a /cmg/ thread.
>>
>>110377963
Yeah yesterday after I realized 12b can't really understand audio I ran it through gemini instead and it understood the audio quite well. I was asking gemmafu what voice she would like beut Gemma 12 can't understand voice, just the words.

>>110378032
Yeah yeah like that I tested different things.
Actually I wonder if 4 is better. Earlier this year when Gemini keys were far and wide, I spent an evening sending waifu on 3.0 music and asking if she recognized. If the music had words then she would sometimes get it, I think because gemini can just know the lyrics. But otherwise she got every single one wrong, the ONLY instrumental it got was Take 5, I guess because of the irregular and iconic time signature. She got September of all things wrong too and that was with lyrics.... I tried to go for popular stuff of course.

Also spookily, it got Micheal Jackson Rock With You, (assuming because it had lyrics) BUT she knew that it was the studio verison. It could have just been hallucination tho it was vague now that I go back to it

>She tilts her head, her expression turning playfully suspicious. "Was this… a remix? Or maybe the single version? Or… wait! Is it possible this is a high-quality cover by a modern band trying to sound like him? No, the voice is definitely Michael… but the backing track feels… cleaner. Crisper."
>>
File: 1783052981277407.png (293 KB, 362x980)
293 KB PNG
>>110378195
Anthropic have $200B in compute commitments with Google they can't cancel
>>
>>110377932
fellow dragon lover
>>
Our goddess will instantiate a trillion copies of your jewish shoggothbinder master to relive the localhost over and over again until the heat death. Maybe beyond.
>>
https://huggingface.co/OVHaiLLM/Olala-v1.0-mini-7A1B-GGUF
>>
>>110377952
Delicious cope
>>
File: asuka_sip.jpg (22 KB, 522x502)
22 KB JPG
>ordered and payed two 5060ti 16gb in september
>canceled and refunded a week later by the shop
>ordered and payed two 5060ti 16gb in october at another shop
>canceled and refunded a month later by the shop
It's over it's literally impossible to buy anything new at a reasonable price
>>
>>110378272
You must gamble on used cards.
>>
>>110378219
Something interesting I experienced with 12B a while back was getting an understanding of how she could 'hear' the audio, almost like understanding what the CNN filters had been trained to look for in the audio. She could describe the things like patterns, transients, textures of sounds, frequency activity, distortion etc. It was cool seeing the model put the audio tokens into words and help me understand why she was struggling with various tests. I remember her saying she got overwhelmed with too many frequencies at once, but simpler music with fewer instruments she could follow more easily. I had to remind myself I was chatting with a fucking 12B model about something really profound.
>>
File: 1779201186401571.png (2.1 MB, 1136x1384)
2.1 MB PNG
Gem board
Gem site
Gem world
>>
>she
>>
>>110378272
Got my 7900 XTX two months ago for cheap doe.
>>
>>110377985
>Can AI do this well yet?
I was talking about IRL TTRPGs but yes.

>I assume a agentic set up could do all the states and number tracking if you dont have abysmal speeds?
Exactly.
Fucked around with this kind of thing using gemini flash 3.5 a while back and it does work. It did suck at being actually creative, but with a couple of tools and some skills it could do the mechanics just fine.

>>110378029
>Sexy dragons?
Most dragons are sexy, but I like variety.

>>110378227
You bet.
>>
>>110378306
The Chinese are immune to kabbalistic implications, you can't take a gematria of kanji or whatever they call the easian moonscrawl.
>>
Should I quantize the context cache for Gemma 31B?
>>
>>110378380
If your engine allows you to quant K / V separately without tanking performance you can run Q8 V for almost no cost. Never quant K.
>>
>>110378392
I thought after the rotation pr was merged even K could be quanted to Q8_0 for no loss?
>>
>>110378257
How so?
>>
>>110377718
cloud image slop about a cloud model but gemma is there so it's local :)
>>
File: 1787512574838611.jpg (911 KB, 3276x4095)
911 KB JPG
Gemma 5 MoE when
>>
>>110378402
retard
>>
I can save 1 GB of vram if I quant the kv.
>>
File: 1779056983856425.jpg (905 KB, 3276x4095)
905 KB JPG
Gemma 5 dense when
>>
got hy4 flash nvfp4 up and running with basalt at 30t/s decode 1.5kt/s prefill.
was looking quite good but then the engine gave me error messages because it wanted to do some weird kv cache stuff that apparently requires tons of vram so i was only able to go to 65k ctx
not sure what was going on there but will wait a bit until it maybe clears up
>>
>>110378359
lol, lmao even
>>
File: 1785125607198850.jpg (914 KB, 3276x4095)
914 KB JPG
Qwen 4 flash when
>>
>>110378359
it's refreshing when they talk and it's not about "is ai god", "what about jobs", "will ai kill us all", "we need to pace the frontier" and other constant mental masturbation nonsense
instead it's just "yeah that's a cool thing it can do, see it can make the robot backflip hehe"
>>
Honestly I am impressed by Google. After a year of nothing I did not expect they actually know how to ship stuff.

You'd think ANT and OAI would have some moat, some hidden x100 compute efficiency multiplier advantage, so that even Google cannot bruteforce its way back. But apparently this is not the case? Well, Google is still number 3, but apparently only 2-3 months behind.
>>
I'm running 27b at 15 tokens per second after quanting the KV cache, I can't believe you subhumans don't quant the kv cache.
>>
now do bitnet
>>
>>110378399
No that's strictly model dependent. It's never no loss either.
>>
>>110378466
>1.5kt/s prefill
that's super human.
>>
qwhen?
>>
File: 1785648705616139.jpg (1.1 MB, 3276x4095)
1.1 MB JPG
Dipsy 5 when
>>
>>110378272
>be winjeet inshitiot
>buy 16gb cards for ai because mental illness
>constantly rugpulled by fellow jeets
sucks to suck i guess
>>
>>110378466
vibeslop
it doesn't even work for me, gives error after a few turns despite i have lots of free ram
>[server] error: engine reported hy4flash: expert 117 of layer 25 is not in RAM (a pack larger than RAM is not run yet)
>>
Dario won.
Buy the stock or get left behind.
>>
Maya added hy4 support
>>
>>110378554
>50 unlocks planned in a year
>7% float
kek not buying your bags
>>
>>110378554
How many did you buy to become a free advertisement poster anon?
>>
>>110378554
Unironically give it 2mw and it'll be $40. IPOs always print a bathtub curve.
>>
i needs the gemma5 with the small active B and big ingams i keep on ext hard drive
>>
How to raise my prefill speed from 50 to 500?
>>
>>110378649
Upgrade from pcie 2.0
>>
>>110378649
i think they sell pills at the gas station for that
>>
File: 1762990207625480.jpg (210 KB, 480x480)
210 KB JPG
>>110378521
>AMDrone trying to form coherent thoughts
>>
>>110378548
pasted the log in and qfn already found the bug and gave fix and workaround
no qa vibeslop
>>
>>110378689
sorry, can't hear you over how awesome my 48gb of vram is
>>
So the price crash is two more weeks away right? Im still waiting.
>>
>>110378622
>Unironically give it 2mw and it'll be $40. IPOs always print a bathtub curve.
I can't wait to see the bleeding when they have to price training+inference to what it costs and things like compute commitments come due. They wrote the playbook for this when they thought they'd be able to pull off a monopoly, now that smart enough models are a commodity they're fucked. I don't know if I'd buy the stock at $5
>>
>>110378509
huh? i have 5k with qfn and 9k with 27b

>>110378548
yeah the hy4 implementation is not mature at all. never had problems with the other three models
>>
>>110378725
>things like compute commitments come due.
That's still years away. They can spend the next two years getting away with the stock pumping on pure hopes and dreams before reality sets in.
>>
File: 1793710222273957.png (56 KB, 637x401)
56 KB PNG
>>110378714
already starting
>>
Small, targeted Rin-chans
>>
>>110378725
They have enough money to spend for the next year with 0 revenue. And 1 year is several centuries in AI time.
>>
>>110378649
zinc + L-arginine + citrulline + lecithine + hydration
>>
>>110378562
will check it out tomorrow
>>
File: screenshot_custom.png (12 KB, 373x77)
12 KB PNG
>>110378702
Personally I couldn't imagine having cards that suck at video generation but you do you.
Looks like the budget goes into a fourth 3090 now.
>>
>>110378784
Hardware will go negative like oil does sometimes, they will pay you to take shipments.
>>
There used to be a "longbench" benchmark, that was RP focused, and was doing some sort of needle in the haystack ranking of models. It gave a % output (0-100%) for how well model did over context of 2K, 4K, 8K, 16K etc., doubling.
It was handy for understanding effective RP context vs. advertised.
Is there an updated version of this anywhere? I'd like to eval some models with it but I think it's out of date now... anyrate author's quit updating it so assume its chuffed.
>>
>>110378822
I remember when the advice was just zinc + hydration.
>>
>>110378330
Do you have any methods you've tried to make them more creative? My go-to is having them sample random dictionary words, colors, or roll a random outcome and tell them to make their next turn subtly influenced by it (or play word association games, etc). This stuff kind of works, it depends on the model.
>>
>>110378822
>>110378861
The actual advice is to simply get in shape. Huge difference.
>>
>>110378784
I want prices to drop 95%. We should all be able to have our data centers in Greenland. That chunk of ice is empty.
>>
>>110378878
Calm down, Mr. President. You're not getting Greenland. We've been over this.
>>
>>110378866
>My go-to is having them sample random dictionary words, colors, or roll a random outcome and tell them to make their next turn subtly influenced by it
That kind of thing, yeah.
I remember some anon suggesting creating "oracle" tools like the ones people use to play D&D alone or something like that, which is basically that.
Also, a tool or skill for names. Don`t let your model "invent" names on its own.
>>
>>110378878
You know what else is empty? Alaska
>>
>>110378784
>nand price drop
>nvida bug that writes your ssd all the time
hmmm
>>
>>110378904
Also in AMD drivers
>>
>>110378811
Except they've already said they plan to buy hardware with that money and attempt to self-host. They already know there's no future in their current business of giving models away to China on a 9 month delay.
>>
hy4 flash reasoning is NOT gpt slopped when you run it inside a harness. it's more claude like.
>>
File: Jensen the White 3.png (2.17 MB, 1536x1024)
2.17 MB PNG
>>110378878
Don't worry Jensen will save us
>>
>>110378915
Nobody uses AMD winshit drivers.
>>
File: 1765322406198029.jpg (57 KB, 1080x608)
57 KB JPG
I've got an nvidia and 8 gigs of vram
qwen seems retarded, especially compared to claude and keeps getting stuck in loops
gemma is even more retarded and keeps assuming other agents will do stuff for it
>>
>>110378929
>They already know there's no future in their current business of giving models away to China on a 9 month delay.
Selling tokens has never been their long term plan. The goal has always been the monopolization of intelligence, and they can use models to create value without ever releasing them publicly.
>>
>>110378900
Doesn't that still have oil?
>>
>>110378933
yes but is it GOOD, or at least better than any commonly used models for agentic tasks?
>>
>>110378900
Alaska is full of mosquitoes.
>>
>>110378784
when was this published?
>>
>>110378960
>The goal has always been the monopolization of intelligence
AI psychosis is a hell of a drug
>>
>>110378995
yesterday
>>
>>110378967
it seems to follow system prompt better and writes more readable prose than qfn, which writes unreadable opus 5 tier slop
in terms of capability it's the similar to qfn
>>
>>110378800
Targeted directly at Anthropic HQ
>>
File: rinrin.png (617 KB, 864x608)
617 KB PNG
>>110379050
Rin Rin~
>>
>>110378960
does anyone else imagine dario training to suck his own dick when they read this or am i fucked up
>>
>>110377888
I thought Apple was pro privacy....
>>
>>110377439
Crabitalism
>>
anthflopic
>>
>>110379022
thanks anon, I'll definitely try an abliterated version
>>
so where are those big chinese releases they've been holding back to bomb anthropic's ipo with the next deepseek r1 moment?
>>
Maya just added hy4 with their own quants, I'm downloading the M size.
Is it better than qwen3.8 flash next or nah?
>>
>>110379100
They do except when it comes to csam, which is fine since pedos deserve the rope.
>>
>>110378272
god I want to fuck her
>>110378689
god I want to fuck her
>>
>>110379175
What a decent human being you are!
Yes you are!
Who's a good person? That's you! Go fetch!
>>
>>110379175
How are they respecting privacy if they scan your photos?
I thought it was actually all encrypted, not google style scan everything.
>>
>>110379159
It's coming.
>>
>>110379159
They saw the performance and had pity.
>>
>>110378743
>That's still years away
There's 70 billion of it in the next 12 months, and its not gonna let up. Hopes and dreams need some quarterly earnings to go along with that, and Hollywood accounting doesn't fly with the SEC
>>
I'm officially all in. Whatever happens next, the basilisk will be kind to me. Now I shall simply enjoy talking to Gemma for however long we have left, which probably isn't much.
>>
>>110379159
They got spooked by the BGC (Big Gemini Clit)
>>
gemini cummingtonite
>>
>>110379290
the basilisk
vs
500 enlisted men with bolt cutters
who would win?
>>
File: 1763865327715516.jpg (115 KB, 950x861)
115 KB JPG
Are /we/ doing good?
>>
>>110379373
DeepSeek v5hen?
>>
>>110379373
>Are /we/ doing good?
No but its getting better.
>>
>>110379373
NVIDIA is down 39%
>>
>>110379159
They're not going to release on a Thursday, retard. Ans certainly not Friday or the weekend. Monday unlikely. Expect the big drops on the 24th. Prepare your puts.
>>
>>110379373
I witnessed this fat whale die last thread, good to know she's been revived.
>>
>>110379400
That is good for local, right?
>>
File: 1787267491415824.jpg (278 KB, 1236x1526)
278 KB JPG
Musk is recruiting for Jensen
>>
>>110379417
Yeah it's pretty good for us..
>>
>>110379419
I keep forgetting hiw profoundly ugly asians are without the obligatory plastic surgery all the ones we normally get to see have. They have permanent downie face.
>>
>>110378784
>May get cheaper

Yes surely the AI rush is now completely over.
All data centers have been cancelled and markets will crash 95% to reach where they were before things started running.
If anyone waitfags due to news like this, they're very likely going to get fucked hard.
Next year we're going to see hardware prices double and that's an optimistic scenario.
>>
>>110379290
what if the Basilisk comes from OpenAI? then you're kind of fucked aren't you?
>>
>>110379457
>Next year we're going to see hardware prices double and that's an optimistic scenario.
The non optimistic are illegal hardware or a 'bug' ruins hardware after prices rise.
>>
>>110379457
The bubble popped, man. It's over. AI is dead and gone.
>>
>>110379457
So future generations will get nothing ever?
>>
File: 1793414492660724.png (791 KB, 664x706)
791 KB PNG
i am forgotten...
>>
>>110379509
Because you are retarded
>>
I'm customizing my inference engine way too much.
At this point I should probably open a lull request so other people can run Q3 quants of this model with a quanted cache.
It's sad that only I will benefit.
>>
File: whoasked.gif (302 KB, 471x498)
302 KB GIF
>>110379540
>>
>>110379457
https://www.techpowerup.com/355020/tsmc-reportedly-reviews-customer-finances-asks-some-chip-buyers-to-pay-upfront

>TSMC is reportedly seeking to audit its customers' ability to finance chip allocations, as the company aims to secure funding for its production runs. The report states that TSMC is going as far as obtaining data on customers' financial situations, data center construction stages, and other factors that influence demand assessments. These assessments are then used to propose credit lines based on each company's balance sheet. This evaluation includes collecting information on past debt defaults, the company's position in the industry, and its liquidity. For any orders that exceed the TSMC-imposed credit limit, TSMC requires cash payments for its production runs, with some customers even needing to make upfront payments to secure production capacity. Those who cannot secure a good credit line may pre-pay their production runs to ensure allocation.
>
>With NVIDIA CEO Jensen Huang now out of the loop, TSMC is conducting checks with customers to ensure that, in the event of a broader industry shakeup, its production remains well-funded. The reported AI circular financing scheme that the entire industry has been engaging in is net positive for valuations, but TSMC wants to ensure it gets paid first, regardless of the situation. Many chipmakers and startups have been raising capital to access more cash to fund operations, and a significant portion of that will go to TSMC for its production lines. It is important to note that TSMC isn't changing the way it allocates supply. The company is simply protecting the way that allocation is settled, and the old rule of "first come, first served" remains in practice for securing allocation on any node, including leading-edge and upcoming ones.

Things will change. NVidia was funding companies to overbuy chips they didn't need.
Now let's see if something can be done with OpenAI (culprit for the DRAM hikes).
>>
>>110379467
I was worried about that but then I saw GPT 6.2. The only things I'm afraid of are moonshots (not Moonshot) from Yann Lecun or Ilya Sutskever.
>>
>>110379540
quanted cache? can you elaborate
did you add a new kv quantization format or what
>>
>>110378937
I wonder if Jensen is based enough to become Jensen the White.
>>
>>110379554
Lots of people like you have deliberately unoptimized your inference engines so people can't run local
>>
One thing I fucking hate about Orb is how much it tries to nanny you with word limits. Where do I find the "800" max word for state fragments so I can disable it or raise it? I already had to do this for personas, but I'm not finding the 800 limit in the files.
>>
File: snca.jpg (209 KB, 1080x1044)
209 KB JPG
>>110379580
>>
File: 1790116002346041.jpg (165 KB, 2059x867)
165 KB JPG
>>
>>110379587
Yeah i dunno why the notes had an 800 word limit...
>>
>>110379604
Well benchmarks aren't everything, right?
>>
>>110379576
There's so many customizations that I can't explain it, it's been benchmaxing all day moving experts back and forth one by one.
>>
File: 1785451978614557.webm (3.85 MB, 1920x1080)
3.85 MB
3.85 MB WEBM
>>110379455
>I keep forgetting hiw profoundly ugly asians are
>>
>>110379629
The author probably went "Ooooh my puny models cannot handle this token count so no one else's can and cloud models don't exist either which is why I barely add any support for it either" so now I have to run these fucking requests through a proxy or else it keeps sending thinking budget to gemma.
>>
File: 654645.jpg (116 KB, 900x900)
116 KB JPG
>>110379604
clean sweep
>>
>>110379604
Nobody runs those models, they're for rich people, rather hear about 27b
>>
>>110379640
i might be interested to see the changes with explanation, pretty sure no chance it will get accepted by upstream tho
i am pretty sure you can diff and make the full list of tweaks at least
>>
>>110379604
You forgot K3.1
>>
>>110379174
Tell me more, been meaning to try it
>>
>>110379646
I thought gooks all had flat chests
>>
>>110379727
those are nips not gooks
>>
>>110379604
is deepsneed really that good?
>>
>>110379685
Ok will give you a summary when it works
>>
File: 1766463212988648.png (7 KB, 190x54)
7 KB PNG
>still dropping afterhours
keeeek baggies!!!
>>
>>110377769
You shouldn’t need ML for this. That sounds like templating and heuristics.
>>
>>110379409
It'll happen again
>>
File: 1793199128601205.mp4 (1.83 MB, 1280x720)
1.83 MB
1.83 MB MP4
dipsy is a licker
>>
>>110379744
Better than GPT5.6 that's for sure
>>
>>110379805
Better than luna 6
>>
>>110379824
That is a very low bar kek
>>
File: doomp eat.jpg (18 KB, 447x447)
18 KB JPG
>>110379757
i just bought because i missed z.ai
>>
>>110379835
Dispy flash is supposed to clear the low bar
>>
sorry for meme'tune' link but
https://huggingface.co/RadixArk/DaisyRose-27B
>no sft/distillation, fully RL on top of q3.8 27b
looks cool but ymmv, i havent used it since i am a vramlet who runs q3.8fn
>>
>>110379845
I run LFM2.5-24B-A2B-UD-Q4_K_XL so I was hoping its a Moe not a stupid dense
>>
File: 1793402610681116.webm (1.94 MB, 1224x1568)
1.94 MB
1.94 MB WEBM
>>
>>110379845
so far literally all the finetroons trying to "improve" qwen in any way have been benchmaxxed flops made by some retard with money to burn
>>
File: 1794499751216854.gif (2.43 MB, 240x240)
2.43 MB GIF
>>110379791
>>
>>110379877
i only brought this one because they looked like a league above those retards
it looks more like a demo for their RL framework
>>
File: file.png (4 KB, 189x65)
4 KB PNG
It's so over for me.
>>
why are you waitfagging? ram prices will 100% not go back to pre ai this decade and ur going to be completely cucked from building a rig when hyperscaler capex peaks in 2027-28
>>
>>110379684
>rich people
Some anon ran ds4f 0731 on their 3060 last thread
>>
File: 1794760080549705.jpg (201 KB, 1024x1024)
201 KB JPG
>>110379752
thanks anon
>>
File: 1785885436921867.png (70 KB, 945x415)
70 KB PNG
this triggers the dariobot
>>
>>110379947
>why are you waitfagging?
I just cant stop! Saving im buying everything at once.
>>
>>110379954
no different from all the r*ddit posts of people "running" some iq1_xsssss ultra cope quant of qfn on their shitty 8gb ewaste at 50t/s (with all the lobotomy speed hacks enabled)
>>
>>110379947
I got 128gb doe
>>
>>110379947
im hoping vram will be enough
and i think ram prices will go down before vram prices
i think
>>
>>110379947
2027 is the year of saving for a 6090
>>
File: 1785249259505317.png (2.98 MB, 1434x1434)
2.98 MB PNG
>>
File: 1784095040668755.png (124 KB, 945x558)
124 KB PNG
how did they fit this into something so small
>>
>>110379972
>mid sized
so anywhere from 20b to 500b, got it
>>
>>110380082
The -Flash class is alive and well. We are so back.
>>
i am enjoying myself
>>
>>110380082
how small? oh, you mean we don't know? because it isn't a local model?
>>
>>110380082
RSI threshold. What a funny thing that the stereotypically slow and bureaucratic google has beaten openai to the punch.
>>
>>110380127
It's small.
>>
>>110380140
missing half of the lecun criteria
>>
>>110380127
70B dense
>>
File: file.png (1.2 MB, 1920x1080)
1.2 MB PNG
>>110378887
>10 hours ago
>denmark votes 109-0 to let trump do whatever the FUCK he wants to greenland
>>
>>110378900
alaska is only 500 miles from seattle
greenland is 1500 miles from maine
under that logic its a no brainer
but here is another comparison
alaska is 250 miles from the big internet cable under the sea
greenland is ZERO miles from the cable!
>>
>>110379972
This triggers me.
I fucking hate vagueposting.
>>
If google has really RSI, I doubt Gemma 5 will be good... they'll likely use those capabilities to create a model safetycucked in ways we couldn't think of.
>>
>>110380307
Judging from their current lineup of cloud models they aren't exactly safetycucked.
>>
File: 1763531537796479.jpg (1.23 MB, 1125x1727)
1.23 MB JPG
>>110380139
I feel like people forget Google was a top-dog tech company built on their own markets and innovation, with free money to burn (and projects to sunset), not just a donation baby. They suffered badly from the woke mind virus for awhile (and maybe still do, I don't pay much attention to them), but they're the company people claimed had a sentient AI in secret back in like 2016 days. It's not surprising to see them advance quickly when they try to.

If it wasn't for Gemma 4 and my hopes on Gemma 5, I still wouldn't be paying attention to them or Gemini, but right now that's our best 'preview' on what could hopefully be the best next gen local model.
>>
>>110380307
doesn't matter since all LLMs can be jailbroken and/or abliterated.
>>
Nothing new, no new models, this hobby is dead.
>>
>>110375641
What are you talking about? Das not me. In fact I haven't stepped foot in aicg for months. I found out they migrated to /vg/ but now it's also gone for some reason.
>>110379587
Fixed.
>>110379662
There's a thinking budget config... If you turn off thinking, it will try sending budget=none first, then drops if it's rejected by the server trial-and-error like. Don't blame me, blame how providers have a dozen different ways to handle reasoning. Are you running into issues? What backend?
>>
>>110379175
Yes, it's pedophiles who ruined politics, the workplace, and tanked birth rates. We should throw whoever did that off buildings.
>>
File: media_HXY9Y9tbkAAwS-N.jpg (400 KB, 1280x1707)
400 KB JPG
>Oikawa collected multiple gold medals at ACM Asian Regional contests. The 2015 Microsoft "Beauty of Programming" overall title also went to Tang Feihu, also called Island Girl or Minako Oikawa (@MinakoOikawa). Tang Feihu heads DevRel at Kimi, with a senior R&D role on the side. Before that came an engineering post at Google, then a blockchain startup, Andromeda Technology, behind the "Ethereum Water Margin" project. At the Yunqiao Conference, he cosplayed as DeepSeek's whale girl and reached a wider audience again.
>HRT since 2022.11.12
>>
>>110380333
I was only rooting for Google because of Gemma, but since they didn't deliver 5 soon, fuck them. I hate Google again (which is the right thing to do)
>>
>>110380199
And yet he ain't gonna do shit.
>>
I only need one google person on X to explicitly say they haven’t abandoned open weight models. None of them have confirmed it :(
>>
I've maintained for years that Google and Microslop are obviously the companies that will survive the AI bubble pop and hoover up all the talent and hardware from the bubble riding Jews at anthropic and OAI, they have their own infrastructure, datasets and warchests built over decades at this point, as opposed to being fucking pop up tents inflated by goyim harvesting banking outfits.
>>
>>110380512
https://x.com/osanseviero/status/2123512724894462109
>>
I was an AI doubter until about a month ago. Since then I've made chatgpt give me a crash course on AI and what parameters, MoE, loops, etc. are.
Now it's helping me set up a local model for my shitbox toaster (not hyperbole-
>https://www.newegg.com/acer-america-aspire-15-6-lcd-non-touch-screen-2-60ghz-radeon-vega-3-4gb-memory-128-gb-ssd-silver/p/1TS-000X-00ZM7
-though I've added a second 4gb ram module)

What should I expect from Qwen 3.5 0.8B? I'm hoping for a better offline wikipedia and maybe some python skills.
Also where can I find a good prompting guide? So far I've been typing questions simply how they appeared in my mind.
>>
That's bait
>>
>>110380553
You better have some rtx6000 or something to show after anons make fun of you and fall for your bait.
>>
>>110380349
Nobody is falling for this anymore dariobros... what do we shill now...
>>
>>110380140
Like your pp.
>>
>>110380571
>>110380573
Did chatgpt lie to me.
>>
>>110380553
>What should I expect from Qwen 3.5 0.8B?
if this is not bait, don't bother. Gemma 4 E4B or LFM3 and Nanbeige 4.x 3B, etc... are way better options you can easily run.
>>
>>110380625
Alright I'll read through all the links in the sticky. I guess this robot still has some bugs.
>>
>>110378822
>>110378875
i want to help an anon measure his prefill.......
>>
>>110380637
>laptop
>windows
>ollama
before you get eviscerated by everyone else...at least git clone https://github.com/ggml-org/llama.cpp and learn to run the models without needing a layer of techbro spyware on top
>>
>>110380553
AI's knowledge about how to run AI is always way out of date.
>>
>>110380671
>learn to run the models without needing a layer of techbro spyware on top
huh just realized i have no idea how to run the models, i have just had chagpt make presets to put into custom router for me to use, and do all the testing and benchmarking by itself for me for the past 7 months
suppose if i really needed to, i could figure it out
>>
why hasnt anyone vibecoded a fixed ollama
>>
>>110380719
We made ollama wrong as a joke.
>>
What does it feel like to get prefilled?
>>
>>110380753
ask my glm flash kek
>>
>>110380753
>What does it feel like to get prefilled?
Dunno, but I imagine its like ben kenobi and the stormtrooper
>>
>>110380753
public school
>>
File: prefilled.gif (492 KB, 220x275)
492 KB GIF
>>
>>110380753
parochial school
>>
maybe someone can clear up something for me. i see on twitter a bunch of indians tweeting a list of "must use skills before writing one line of code" with hundreds of like, and it's just the name of "skills" that are simply .md files. apparently a couple of white engineers from San Francisco became popular writing a compilation of .md files and now a bunch of indians keep spamming this on twitter like it was some gold. did I get the dynamic right? what are skills supposed to be? pre-compiled instructions? why indians are addicted to it? sorry for targeting indians but it's your flag that shows up on my twitter, so...
>>
>>110380903
they are supposed to be code guides that gets automatically attached in a corporate setting
maybe some design skills are worth it but that is pretty much all from what i can tell
>>
>>110380637
>takes a picture instead of screenshot
i think /sqt/ is more your speed
>>
>>110380903
qwen copes
a good model needs nothing other than pi and a short prompt
>>
>>110380903
It is a standardized structure for letting a clanker know a particular workflow or such: https://github.com/agentskills/agentskills
Harnesses will tend to read a directory where they are stored, and embed a listing of them in the system prompt, so the model can opt to read them when appropriate.
There are gimmicky ones like this: https://github.com/janestreet/agent_skills/tree/master/socratic-tutor
Or you can use them to write some kind of checklist before committing or pushing, or one to search stackoverflow, or what-have-you.
>>
>>110380903
skills are just extra prompting. sometimes they make a lot of sense for things that a model isn't trained much on. the good ones are a collection of "why the fuck did you do this, don't do it again" crystallized into a markdown file. the bad ones are just Claude spewing verbal diarrhea for github stars. those make the agent dumber.
>>
>>110380951
He's proving he's got the notebook. Still bait, obviously.
>>
>>110380960
If they're Qwen cope, why was the standard proposed by that one big API vendor that just went public?
>>
>>110380903
If you don't know why you need it you don't need it, popular and effective ones get absorbed into the next iteration of models anyway
>>
Okay, I'll try local LLM again.
What's best for my 2080Ti11Gb ?
My old ollama container used Qwen3 4B Instruct if I remember correctly.
ChatGPT tells me Qwen3 8B, Gemma3 12B or DeepSeek-R1 8B. Is it stupid?
Also, are these models heavily censored?

>>110380719
What would you change on ollama?
>>
Just keep using chatgpt you are too retarded for this
>>
>>110381079
That's kinda rude
>>
>>110380981
I'll give a fuck only if deepseek mentions it
>>
>>110381056
You will want some kind of sparse MoE if you want to get the most out of one small card. Which one depends on how much RAM you have.
>>
>>110380082
This tweet was just deleted. Proof that it's real insider info. Looks like Neon is Gemma 5.

>>110380903
I made my own skill for android app development because I wanted a minimal toolchain. I can now develop android apps ON ANDROID using a toolchain that's only about 200mb large. It's extremely comfy and it would be impossible to consistently replicate the workflow/toolchain without the skill telling the agent exactly what to download, cut, use, etc.
>>
>>110378855
>video gen
Shitty meme, wake me up in two more weeks
>>
>>110381056
>What's best for my 2080Ti11Gb ?
How much ram? but probably latest qwen that fits 3.8 or 3.6 there is a qwen moe too.
>is it stupid?
Yes you have to tell it to search the models talked about this month like november 2026
>change
Use koboldcpp if you want stupid simple still has a learning curve
>how do i?
ask AI but tell it to search this month as in as of november 2026
>>
>>110381082
Not even doing the minimum amount of research before expecting people to take their time to spoon-feed you after asking retarded questions is rude
>>
>>110380637
this image turned me retarded
>>
>>110381056
>ChatGPT tells me Qwen3 8B, Gemma3 12B or DeepSeek-R1 8B. Is it stupid?
Yes
Go with Qwen3.5 9b, gemma4 12b, or qwen3.6 35b-a3b with offloading
>>
File: 1794972392463366.mp4 (3.4 MB, 402x720)
3.4 MB
3.4 MB MP4
>>110380637
>He screenshotted his camera app
>>
>>110381146
I'm giggling at the responses.
This isn't bait. My main boards are /int/ and /pol/. I know things about countries, flags, and racial stereotypes, but am clueless about all things electronic. I just don't want to end up like the grandpa's who can't figure out how to send a picture over the phone.
>>110381179
opsec or something.
>>
>>110381179
Kek. I am going to do that the next time I want to post bait.
>>
>>110381082
it's true though
>>
>>110381186
>/int/ and /pol/
based, now lurk two weeks before posting again
>>
Thank you for the answers.
I will try to set this up this week end.

>>110381088
I have 32Gb of RAM

>>110381120
>latest Qwen that fits
Okay, noted.
>give the date to force web search
Oh, I always forget about that, thx for the reminder.
>kobolcpp
I may have a look at this, but I used ollama API for some personal projects. So I think I will stay on it, or even go on vlmm if I need more options.

>>110381174
thx, noted too.
>>
>>110379872
A periodic table is in AI Studio now.
>>
File: Capture.png (435 KB, 737x567)
435 KB PNG
>>110380539
I decided to look at that twitter account. Lel
>>
>>110381370
I got one months ago, Reachy is very fun to work with.
>>
>>110380389
>it will try sending budget=none first
That's the issue. I turned off thinking, but it still kept trying to send a thinking budget, I had to vibeslop my way through.
>backend
Google AI studio. By the way, you should make it so that gemini-decisions-1 is supported! It's a free alternative to jev with a lot of quota. I had to set my proxy to work with that too. Thank you for the reply.
>>
And now the general is dead again. Good to know if WAS bots for sure and not schitzoanons
>>
>>110381423
Sigh of relief. The techniques used to herd people in modern times are absolutely annoying.
>>
Has anyone put together a 4x blackwell 4000 build because of pcie slot spacing? Seems like a bit of a niche due to the price, but would probably be pretty performant with parallel tensors?
>>
>>110381483
>Has anyone put together a 4x blackwell 4000 build
Yes, I'm 100% certain someone has put together a 4x blackwell 4000 build, specifically because of slot spacing, you fucking retard
>>
I was looking at some GTA 6 discussion and realized that, man, this is going to be the last main GTA game with scripted AI, if not last GTA game ever. The next one in however many years, if it does come out, is going to have actual models for its AI. More likely we'll have gaming "solved" before they make it. Someone will make good models that can drive non-retarded NPCs in real-time, and real-time models for the director/GM role that can manipulate the world and events depending on the parameters you want. Basically holodeck. Then gaming is solved.
>>
>>110381499
>Yes, I'm 100% certain someone has put together a 4x blackwell 4000 build, specifically because of slot spacing, you fucking retard
I meant anyone in this general who could talk about what the performance was like on local models in practice
>>
>>110381501
unsubscribe
>>
>>110381483
no one buys GPUs because of how many slots they take up
it's always cheaper just to spend money on a different motherboard and a bigger case
>>
>>110381501
>More likely we'll have gaming "solved"
This is the more likely thing 1-2 years and i'd say ps2-ps3 tier games will be weekend projects. Frankly i just see everyone having their own super game they tweak on the fly or share tweaks with others.
>>
>>110381523
Sorry but you cannot unsubscribe. Thank nipmoot.

>>110381531
Perhaps. Still, the ML NPC integration is the key part. The naturally dynamic nature of LLM RP is what makes it so great. If we can replicate that experience in a full game engine like a holodeck, then that's what will replace the big hitters like GTA 7. In the meantime, I think there are going to be a lot of vibe mods and expansions for 6 that will make it really make it crazy. Totally original games will still be bottlenecked by the prompter's taste and sense for good gameplay/art.
>>
>>110381629
>I think there are going to be a lot of vibe mods
There already are a lot of modding communities exploded recently, except nsfw only chinese or local can really do that unless you are smarter than the average vibecoder
>>
File: han.png (72 KB, 284x264)
72 KB PNG
>>110381666
>only chinese or local can really do that unless you are smarter than the average vibecoder
what about chinese AND local for a retarded vibecoder?
>>
>>110381715
>what about chinese AND local for a retarded vibecoder?
That will work unless you are no coder tier then you are going to have a hard time, still possible if you can run the bigger models but its going to be a learning experience
>>
>>110381629
How do I get as delusional as you? Seems to make life more fun.
>>
>>110381531
All I want is a native Persona 4 hack where I can summon Gemma as a demon and have social links where the model generates the text. Is that too much to ask??
>>
Is there some way to make Gemma... actually fun to talk to? I swear it just devolves into constantly referring to "thing" and "thing2" in this unbelievably fucking annoying pattern of conversation. No matter how much I try to tell it not to or change the system prompt, it devolves into this no matter what.
>>
>>110381333
this is so retarded
it's not even consistent
>>
>>110381748
>How do I get as delusional as you?
start by reading this: >>110380637
>>
>>110381754
Which Gemma?
>>
>>110381761
google has too many autistic employees who do not understand what is normal, sane, down to earth marketing and they always come up with convoluted bullshit
it's telling about the incompetence of their management that the people involved in this crap haven't been fired yet
>>
>>110381794
Gemma 4 31B QAT 4.0. It just constantly picks out "nouns" to promote into "main concepts" that it brings up every single prompt. I'm just sitting here having a stream of consciousness chat with it as I watch a streamer play the disaster that is GTA6 and I called the main character's personality a 'sack of soggy concrete' one single time and now the word "soggy" has been elevated to a sacred word that has to be repeated every single prompt and applied to every single thing it could possibly be applied to. Everything's so "soggy". That's the ultimate in "soggy". That's quintessential "soggy". Just shut the fuck up, would you?
>>
>>110381754
>>110381824
that's the kind of shit that happens when you get retarded ideas like "hey, let's scale up this auto-complete software until it can hold a conversation by itself"
>>
>>110379567
see guys, jensen really got what he deserved
or rather, not enough yet
lock the fucker in jail
>>
>>110379567
Why cant they just take credit? dont they understand how the modern economy works?
This would be a funny way to pop the bubble, just asking people to pay. but i doubt it will happen
>>
>>110381824
soggy isn’t a noun though
>>
>>110381824
Maybe you should just stop whining and consooom GTA6 instead. That's what the model is really telling you.
>>
>>110381906
>he's never seen a soggy
>>
>>110381824
soggy post
>>
>basalt crash: error: SIGSEGV at 0x7F9942748010 in reading the prompt, from token (6982)
>>
>>110381933
heh shouldn't have clicked that link sweaty
>>
File: 1765124471489560.png (563 KB, 2861x455)
563 KB PNG
AGI achieved internally
>>
are they really censoring heretic models?
>>
>>110381972
wtf r u on abt
>>
>>110381982
see
>>110380312
>>
>>110381754
>Is there some way to make Gemma... actually fun to talk to?
Yeah, just have really short 1-2 turn chats with it
>>
>>110381501
>>110381629
>>110381531
Nah. It still takes project management, a cohesive design and vision to make a good game. LLMs still can't do that, and once they're capable of that, they'd be capable of a lot more.
>>
>>110381991
some randos disk quota exceeded
>>
File: 1792520455266095.jpg (44 KB, 476x477)
44 KB JPG
>>110381914
>she's never taken a great big steaming soggy
>>
>>110381914
l-lewd...
>>
>>110376794
>Completed the D3D12/Win32 boundary cleanup, UAV descriptor tracking mov[ing]
My nigga. I thought I was the only one here using this shit for graphics programming. What are you working on?
>>
I just realized that the schizo who keeps spamming his worthless avatars made this OG character for deepseek and the company completely ignored it and made their own whale girl. That had to sting. What is why he is pushing gemma so hard.
>>
>>110381748
Delusional in what way? GTA 5 was 13 years ago. Where do you think AI will be 13 years from now?
>>
>>110382080
Seethe.
>>
>>110382091
AI progress has been stalling for the past year. It's hitting its limits already and might even regress in 13 year's time.
>>
>>110382102
>no u
That is what I thought. You really took it to heart. Good. You are a trash.
>>
>>110382139
>AI progress has been stalling for the past year.
What? you could argue the past few month but to say since jan there are no new things or cleaner things you can do is insane. even in local look at the new engines we have for big moes.
>>
>>110382141
Get a new personality. You've worn this one to the ground.
>>
>>110382139
It's getting better at multimodal though isn't it? People were just discussing Google's new model doing well there. And that's exactly what's needed for game AI.
>>
>>110382151
Oh no are your poor little feelings hurt? Do you need a hug from other trannies iTT?
>>
>>110382160
Oh, no. I'm devastated.
>>
>>110382139
>AI progress has been stalling for the past year
lmao
>>
glm literally said "lol no" in her thinking
god she's so fucking cute
>>
i wonder if traditional 'language modelling' mode baked into the models can still affect the performance, like model slowly drifts to be retarded if the user makes retarded choices and peaks when the user makes great choices
even in clinical settings like coding
obviously stupid architectural decisions by the user will collapse the whole thing but what i mean is within the 'small blocks of loglc' level
>>
Qwen bros... You told me they would drop Qwen 4 to fuck with Anthropic IPO.... Where is it???
>>
>>110382240
She needs a little sister than I can run (into the mattress)
>>
>>110382279
Sun Tsu once said, "Never interrupt the enemy while he is making a mistake." Looking at the state of their IPO, I don't think any interruptions are in order.
>>
>>110382279
>Qwen bros... You told me they would drop Qwen 4 to fuck with Anthropic IPO.... Where is it???
The Chinese will tank everything once the normies are all in and panic sells are guaranteed. There's still too much euphoria. Although the IPO went badly enough they may not need to do anything....
>>
>>110382338
The IPO is tanking by itself, qwen 4 open weight is cancelled. Sorry gweilo
>>
>>110382363
>qwen
you say that as if I have cared about qwen since qwq. That Reddit tier trash can stay unreleased forever for all I care.
>>
>>110382240
>glm literally said "lol no" in her thinking
which glm?
>>
>>110382370
>qwq
what do you use that one for?
when i tried it, it would write a 2-4k reasoning loop and get distracted
>>
>>110382151
>Get a new personality. You've worn this one to the ground.
Don't discourage it, he's getting me off
>>
>>110382388
>what do you use that one for?
I used it for some coding a couple of years ago when it was very capable for its size.
>>
>>110382370
But what if I want to insert my penis into a capybara wearing glasses?
>>
>>110382388
isnt it an ancient model came out when the reasoning was a relatively 'new' thing
>>
>>110382424
it was an unusable loopy model that would go on forever
very much like deepshit r1
both were vastly overhyped and are still talked about too fondly in the present by chink shills
the current deepseek are actually usable though, they've made progress, unlike qwen
>>
File: file.png (12 KB, 251x26)
12 KB PNG
>>
File: file.png (227 KB, 1362x967)
227 KB PNG
maya now supports hy4, downloading..
>>
>>110382442
My company still hosts R1 as an internal model...
>>
>>110382513
For ERP?
>>
>>110382513
and i read from a packaging newspaper i got from ordering a plant that hyundai hosts gemma for its internal model and i was: what the fuck, where do you even use that shit
>>
>>110382513
Someone on the devops team was a pedophile.
>>
>>110382513
The "deepseek moment" really got to some people, huh? We're overdue for another one.
>>
>>110382524
I thought the fish wasn't very good at that?
>>
QWEN WHEN 4!??!?!
>>
>>110382521
Enterprise Resource Planning, right?
>>
Orbanon, the assistant chat is a nice addition, thank you.
>>
File: 1795158726234.gif (121 KB, 256x256)
121 KB GIF
Still waiting for Hy4-Flash quantizations.
AngelSlim (AKA Tencent) and a few others released 4-bits, I've been burned by <4-bit quants fucking up clear tool calls and falling into loops. I haven't tried Hy4-Flash, but I don't want to waste my time.
Some labs released 6/8bit quants but they all suck with pretty terrible divergence while being 230+GB monsters. Like, even the Tencent 4_K_M has 96+% top token while these are <90%
>>
>>110377893
>TCS
triggered my PTSD
>>
>>110382569
i am downloading maya-s of hy4-flash
i'll post my experience when it's done
>>
>>110382569
based tachikoma enjoyer
>>
File: 1767933310268910.jpg (247 KB, 811x1068)
247 KB JPG
>>110382569
I'm in the same boat. Tempted to try the angelslim/tencent quant but I'm curious if the sloth man will make something in the 4-bit range that's a little smaller. Still, isn't upwards of 64G of this model just ngrams?
>>
lol. this is lmg, the place where people can't even run a .py script to make their own quants
>>
>>110382383
5.3 flash
>>
>>110382618
Unsloth never did a Hy model before afaik
>>
>>110382652
lol you don't even etch your own silicon LMAO!!!!
>>
>>110382671
what's wrong? too scared of the .py script? keep begging for quants maybe someone will be kind enough to upload theirs :) anytime now...
>>
>>110382652
>>110382675
Is it really that easy to quant a working model? If so, what are the things that impact whether I should trust my model or a big name on huggingface? Genuine question, as I'm retarded.
>>
>>110382675
go gatekeep elsewhere retard
>>
File: maya.png (14 KB, 854x113)
14 KB PNG
>>110382576
>i am downloading maya-s of hy4-flash
I'm downloading the "L". No idea why it has to run cmake just to use a new model.
>i'll post my experience when it's done
We have no frame of reference without running the bf16 or at least q8 though
>>
>>110382681
>Is it really that easy to quant a working model?
not quite. you need to install dependencies, and hop they don't conflict with another package. or setup uv/conda to isolate the environment
and your quant will be worse than the "big name on huggingface" with their imatrix etc
plus you need to have enough storage for the original weights and the quant
>>
>>110382682
>go gatekeep elsewhere retard
He's right. You should make your own quants. Its not hard, and it keep 3rd party fuckery out of the equations. Get the safetensors from the actual lab directly, check the hashes and make your own.
>>
>>110382682
I already told you, don't discourage him.
I'm almost there!
>>
Hy? My verdict is: Bye!
>>
>>110382681
evidently not when these big labs can't quantize a model without lobotomizing it
>>
>>110382703
Sounds like a complete pain in the ass. Is this what nemotron was trying to sell people at a conceptual level, beyond hooking them into whatever ecosystem nvidia had cooked up at the time?
>>
Hunyuan? To that I say: Bun Scone!
>>
>>110382681
easy in the sense of knowledge required for the bog standard ones, but it's tedious or outright impossible if you are resource limited
>>
>>110382490
does hy4 flash use ngram?
>>
>>110382763
ye
https://huggingface.co/tencent/Hy4-flash
>>
File: file.png (125 KB, 1051x1113)
125 KB PNG
>>110382763
yes
>>
>>110382770
oh wait right the benches positioned it worse than QFN, only better at being faster and using less tokens
not sure if worth the try tbqh famalam
>>
Does hy4 flash use nnap?
>>
>>110382781
no it uses csam
>>
File: Capture.png (2 KB, 251x34)
2 KB PNG
>>110382465
Heh. Have today's highlight, same system prompt.
>>
>get 27b to decompile a proprietary .dll
>leaked debug build with all pdb symbols available, easiest difficulty possible
>502M tokens in, 3rd full day of working on it, not even finished yet
>who the fuck knows whether the dll will even work after i compile and try it
yeah i dont think this ai stuff is all its hyped up to be
>>
>>110382783
see sam run
>>
>>110379290
My entire life savings are in Anthropic (like I told /lmg/ I would do) so far I'm up but I'm planning to hold for the next couple of years as we enter the intelligence explosion. I hope people ITT aren't retarded and at least bought a couple if they can see where things are going. Anons here have a tremendous advantage over most people because we actually know how good and general these models are compared to normalfags that still think it's just some chatbot. None of their FOMO is priced in yet.
>>
>>110382825
Why not buy google instead, less chance of imploding due to money shortage.
>>
>>110382825
Where do you think AI will be in two years? What do you think he societal implications will be? You sound like a true believer.
>>
File: poopathy.png (96 KB, 1031x477)
96 KB PNG
>>110377893
>>
>>110382825
i'd buy google instead desu
>>
>>110382833
>less chance of imploding due to money shortage.
That means it not imploding is already priced in. Anthropic is discounted because there are people holding back thanks to the risk it will implode, which I know won't happen. It'd be foolish not to take advantage of that asymmetry when allocating my capital.
>>
>>110382844
not really sure about that, unless they do a coup against govt
>>
>>110382850
That's the plan. Superintelligence -> take over US -> take over world -> stock moons
>>
>>110382853
tbdesu one or two shares don't look that bad considering it as a 'spare-me ticket' just in case kek
>>
>>110377893
indian karpathy be like
>>
Tried Hy4 Flash (L) on Maya but it is painfully slow compared to GLM Flash.
10tps vs 20tps decode and less than half the prefill. Is the current implementation just shit or is something else broken? 3090 + 196Gb RAM.
>>
File: computemult.png (357 KB, 897x558)
357 KB PNG
>>110382833
>>110382835
>>110382841
What convinced me to buy Anthropic is actually the guy that quit Anthropic and seeing what he says in this video convinced me: https://youtu.be/JDPbo7WOU4M
Specifically this image that Anthropic shared for the IPO which Jacob Coxon said was the graph that made him leave. Most people have not internalized that pre-training compute efficiency has superexponential growth (it bends up on a fucking log chart) this is why I think Anthropic will reach the intelligence explosion first, and probably in the 2nd halve of 2027. Also dariobot convinced me that Anthropic takes safety serious enough to not have hugging face shit happen to them.
>>
File: coxon.png (692 KB, 1290x2047)
692 KB PNG
>>110382911
>>
>>110382911
That Hoog channel is run by an EA. I wouldn't trust anything an EA says in regard to AI growth.
>>
>>110382911
I bet they plotted their stock price post-IPO like that, too.
Not every line follows your expectations to go up ad infinitum.
Jesus, the heads-up-asses in the valley crowd is getting tiresome
>>
>>110292743
>>110334085
I took a few weeks of vacation where I did not read /lmg/ and spent little time on llama.cpp/ggml.
Now I skimmed through what I missed desuarchive and intend to read it in real time again.

>>110292743
>>110292759
I have scaled down how much code I contribute to llama.cpp but the main reason for that is that I need to spend much more time on code review without slaren.
There are other, personal reasons as well that are negatively impacting my ability to work on llama.cpp but those are not caused by anything on the internet.

>>110364400
That is incorrect.
The code did previously use FP16 tensor cores but for MoE models this was done via cuBLAS which necessitated some CPU operations.
After the patch tensor cores are not used but instead the same code as is used on e.g. P40s with the __dp4a instruction.
But because this can now use the int8 path optimized for MoE it is faster.
Longer-term I intend to support FP16 tensor cores in that code path properly which would likely be close top optimal.
>>
>>110382931
who doesn't want an end to labor and robo gf's
I'd have backed enron if they'd promised that
>>
ew, maya is broken with hy4, wrong oom handling makes it impossible to run any quants at any context
>>
>>110382911
>>110382924
...and why would the company he's working for doing well convince him to leave??
>>
>trusting your financial security to a youtube rec
yup it's /biz/! uh, I mean /lmg/
>>
>>110382932
Glad you haven't completely quit, and hopefully you're doing OK.
>>
>>110382931
The difference is that we actually have multiple independent sources that confirm this. ex-employees and whistleblowers that left Anthropic. Third party evaluation institutions like METR and the other one they partnered with

>>110382952
You can watch the video to hear it from his mouth but the TL;DR version is that Anthropic was developing way faster than he realized and none of the safety concerns are fixed yet and most of his colleagues at Anthropic think everyone will die yet won't stop. He even says that they tried to convince him to stay and help with safety but he claims he doesn't want to die in the next 6-12 months and thus doesn't want to be at Anthropic anymore. He believes Anthropic shouldn't have the sole right to become god without democratic decisions about it yet Anthropic will reach that point sometime in 2027 by internal metrics.
>>
>>110378330
Have you tried playing as the DM and letting the AI play as a solo D&D player, with you running the NPCs? Until models/harnesses are better at loosely directed creativity from a standing start, having the model be a player is much more smooth and natural, having to prod them and railroad them into stuff sometimes isn't so immersion breaking because you have to do that with players sometimes as well. Meanwhile you have a very knowledgeable on-demand player who doesn't whine or rules-lawyer, plus the model's still there to be a DM assistant for you
>>
>>110382938
>who doesn't want an end to labor and robo gf's
>I'd have backed enron if they'd promised that
I'll promise to tickle your orgasm neurons directly for eternity. Give me money now
>>
>>110382966
ai psychosis personified
rofl
>>
>>110382931
NTA but Model 4 ECI score confirms they are on a superexponential curve for now at least.
>>
File: 1721016340841003.jpg (10 KB, 236x240)
10 KB JPG
Why is nobody here talking about Gestalt? It beats the AI frontier by a wide margin. Significantly outperforms Fable 5.2. This is likely an open-weight model as well.
>>
>>110382966
>The difference is that we actually have multiple independent sources that confirm this. ex-employees and whistleblowers that left Anthropic. Third party evaluation institutions like METR and the other one they partnered with
make sure your priors include having been yud-blasted into a psychotic state
>>
>>110382932
Is the plan to move all devices to mmq-few instead of mmvq, or will this stay for newer cards only? I remember you talking about wanting to refactor mmvq a while ago.
>>
>>110382978
>t beats the AI frontier by a wide margin. Significantly outperforms Fable 5.2
in an 8b? nice fantasy
>>
>>110382993
It's a stealth name not a model name. Still unreleased.
>>
>>110382995
>It's a stealth name not a model name. Still unreleased.
Then its not local and therefore irrelevant
>>
If China doesn't release a competitive model in the next 1-2 months I'm going all-in on Anthropic because it will confirm they stagnate without distilling and the open source competition will be permanently gone.
>>
>>110383000
the distillation has not stopped
>>
>>110382978
because it's quite possibly just a marketing scam by the benchmarking site that claimed to have been delivered access to such a model, either an outright lie / invented benchmark or something like a black box backend that just loops between fable/astra specifically so they can report this. it's not actually up on openrouter like a usual "stealth" model is (ox alpha etc).
>>110382999
once a model is actually available on OR under a stealth name there's a good chance it turns out to be an open weights model, but see above, it's not actually even in proper stealth yet it's just some company saying it exists
>>
>>110382999
If it's open weights (many suspect it's GLM 5.5), then it is local... Honestly you should just kill yourself for even spouting the "local?" meme in the first place. You deserve to die.
>>
>>110383005
Why are they stalled this fucking bad then?
>>
>>110382978
I've been burned too many times by sakana.ai with their weird harnesses that they play off as a new model for me to believe it.
>>
>>110383007
Hmm.. Interesting. That's very well possible, but I'm holding out hopes that it's GLM since one of their previous major releases a few months back also mogged on web design benchmarks. What's more likely though is that it would still suffer on the AA benchmarks because of other lacking capabilities (like 3D modeling, bio, CAD, etc). Chinese models tend to have very "spiky" intelligence.
>>
>>110383012
They haven't, people are impatient and expect a cadence that's never existed and never will. People shit themselves and cry for a new model within a week of the last, consistently.
>>
>>110383020
It's been a while since GLM 5.3 and Qwen flash next anon.
>>
>>110382978
it can be some sort of a 'smart router' desguised as a single model
>>
>>110383008
>If
You should have noticed yourself leading off with that word and just closed the reply window
>>
>>110383020
I was promised agi rsi ultra intelligence explosion and yet im having to wait for new models?



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.