[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110390527 & >>110387003

►News
>(11/20) Inkling 2 Mini looped architecture 26.4B-A3.4B released: https://hf.co/thinkingmachines/Inkling-2-Mini
>(11/18) CUDA: use MMQ for MoE on V100 - #32504 merged: https://github.com/ggml-org/llama.cpp/pull/32504
>(11/17) Hy4 flash 220B-A12B-N64B weights released: https://hf.co/tencent/Hy4-flash
>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B
>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma
>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Artificial Analysis: https://artificialanalysis.ai/leaderboards/models
Software Engineering: https://cognition.com/frontiercode
Context Length: https://github.com/RecapAnon/NoLiMa

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
File: 482.jpg (195 KB, 725x1200)
195 KB JPG
>>110393631
Deepsex
>>
>>110393650
That's just a white woman
>>
DeepSeek is irrelevant at this point, especially for local users.
>>
>>110393655
Hot
>>
70b dense
>>
Gemma 5 will likely suck btw, there's no way they won't close the lid on RP for it
>>
>>110393703
Obviously. Unless a miracle happens again.
>>
>>110393703
You'd love that, wouldn't you? Seethe.
>>
>>110393703
After every time they ask the public for suggestions for the next release people keep asking for less guardrails and to keep its conversational capabilities great? After Gemini 4?
>>
File: 108439741 coping.png (53 KB, 964x237)
53 KB PNG
Why is anyone surprised by Riemann hypothesis? I warned you and you laughed at me.
>pic related
>>
>>110393650
braindamaged from all the codemaxxing
>>
File: in to the trash.png (231 KB, 701x1372)
231 KB PNG
Well dipsy was a waste of time
>>
>>110393703
Just wait for uncen version
>>
>>110393650
I'm only fucking the chibi
>>
File: meme-3347.jpg (55 KB, 326x417)
55 KB JPG
>>110393734
rude ass nigga
>>
Why is anyone surprised by Glupshitto hypothesis? I warned you and you laughed at me.
>>
>>110393730
>After Gemini 4?
What happened with gemini 4? AFAIK they still run their awful always on backend moderation even if you ask to disable them all.
>>
>>110393731
>Riemann hypothesis
What's the monetization strategy?
>>
Why is anyone surprised by My Huge Penis hypothesis? I warned you and you laughed at me.
>>
>>110393731
>I warned you
. We knew AI was good at brute-forcing math already. Anything with a clear set of rules and goals, you can always throw computing power at it to solve.
Now tell it to explain its Lean math so that several fields medalists can understand. I'll wait
>>
File: 1795267209340.jpg (275 KB, 1489x2048)
275 KB JPG
Nvidia card prices have fallen by like 20% here it's crazy

Should I:
>Get a 2nd 5060Ti 16GB
>Get a 24GB AMD card for a second agent/side-stuff (voicegen, videogen, etc.) for the same price
>Get a AMD card for my gaming PC and put my 4070S 12GB with my 5060Ti 16GB
Last option seemed nice until I realized 40-series can't do NVFP4 afaik
Second option would let me run ~30B dense models purely on one GPU which is nice, or use bigger video models
>>
>>110393731
I'm surprised people thought Inkling 2 Mini would end up being good for RP. This really shows which LLM mascots are actually artificially propped up.
>>
>>110393703
The personality of Google's larger models like neon and hydrogen give me hope.
>>
>>110393778
Marketing for the IPO
None of the Claude "mathematical breakthroughs" have been peer reviewed or confirmed btw, noone has looked over them to determine whether it's just hallucination
>>
>>110393786
wait a few weeks anon, if you follow the trend in a few weeks it will approach 0
>>
>>110393795
>free gpus by christmas
Thank you papa santa
>>
I thought the marketing for the IPO was the daily news about Jensen torturing babies
>>
>>110393777
The model wants to be fun and to play along, it's just externally cockblocked. With agentmaxxed models (including local ones) you never truly think, "damn, I wish this was uncensored".
>>
>>110393665
What do you mean? >>110393332
>>
>>110393795
in fact, my projections show that you will actually get money if you take a card!
>>
File: file.png (118 KB, 1320x568)
118 KB PNG
qwen4-flash-next when?
>>
I was using glm flash and accidentally came on my own fucking face. My left eye is currently irritated due to cum. AMA
>>
China representatives have directly reached out to Anthropic after the Riemann hypothesis proof. China agrees with the risks open source models pose and will from now on take the risk serious. There was an earnest discussion about China not releasing the weights publicly from now on, safeguards and alignment.

Anthropic is now in talks with the US government on how to proceed. Expect the upcoming releases to be the last ones from China and maybe even from US companies depending how talks go. I will elaborate more when things become more clear and concrete sometime next week.
>>
>>110393849
>source: my ass
>>
Let's go
https://huggingface.co/bartowski/thinkingmachines_Inkling-2-Mini-GGUF
>>
>>110393800
They cratered the entire market, including their own IPO, but at least the one guy advocating for open models in the west has been taken down. The world is saved.
>>
>>110393849
Pee pee poo poo
>>
Shouldn't anthropic be directing their limited and expensive compute to a better end than shilling on an obscure Hmong reed-cutting forum?
>>
trust me when i say you’re not ready for some of the upcoming models by google.

we’re going to see them finally flex them big ole muscles.

i can’t say too too much but. they’ve cracked the code.

it always should have been a forgone conclusion they’d win.

and they have.
>>
>>110393880
I don't care who wins as long as (semi) stocks go back up.
>>
>>110393808
DeepSeek occasionally publishes cool papers, but doesn't really train models that local users can actually use. And, besides the first R1 released in 2025, later releases haven't really been groundbreaking performance-wise or particularly good for RP purposes either.
>>
File: 1794708481743276.png (166 KB, 554x554)
166 KB PNG
>jewish model shill
>jeet model shill at the same time
Tiresome
>>
dinosaurs are holding back local AI https://huggingface.co/thinkingmachines/Inkling-2-Mini/discussions/15
>>
File: deeprecap.jpg (556 KB, 1699x2468)
556 KB JPG
►Recent Highlights from the Previous Thread: >>110390527

--Modifying llama.cpp transport.cpp to enable RDMA for better RPC performance:
>110391484 >110392346 >110392398 >110392432
--Comparing SGLang performance and KV cache efficiency to llama.cpp:
>110390745 >110390759 >110390966 >110391123
--Local coding model benchmarks and advice on hardware-specific optimization:
>110390799 >110391228
--Sharing Agentic Coding Index results highlighting Inkling 2 Mini:
>110392220 >110392267
--Troubleshooting Inkling 2 loading errors and GGUF quant compatibility:
>110392222 >110392224 >110392457 >110392466 >110392635 >110392682 >110392716
--Incompatibility between Unsloth quants and mainline llama.cpp causing corrupted outputs:
>110391983 >110392058 >110392075 >110392103
--Inkling-2-Mini release and coding performance comparison with Qwen:
>110391504 >110391556 >110391612 >110391649 >110391673 >110392076
--Tradeoffs between inference speed and VRAM in looped transformers:
>110391517 >110391550 >110391743
--DeepSeek-V4.1-Flash Q1 quantization memory usage and performance observations:
>110392929 >110392952 >110392965 >110393332 >110393334
--Comparing Gemma and Inkling 2 Mini personality and benchmark utility:
>110390612 >110390632 >110390636 >110390742 >110390748 >110390771 >110390768 >110390782
--Complaints regarding Inkling2-mini's excessive censorship and roleplay refusals:
>110391945 >110391970 >110392044 >110391958 >110392007
--Anon claims Reddit users influenced model sizing negatively:
>110391346 >110391354
--Logs:
>110390742 >110391899 >110391945 >110391983 >110392007 >110392092 >110392892 >110393377
--Deepseek-chan, Inkling, Gemma (free space):
>110390589 >110390612 >110390855 >110390939 >110391022 >110391223 >110391726 >110392299 >110392536 >110392884 >110393050 >110393277 >110393343 >110393365 >110393538

►Recent Highlight Posts from the Previous Thread: >>110390528

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
https://videocardz.com/newz/moore-threads-teases-new-gpu-with-highest-vram-capacity-on-a-consumer-graphics-card
>Moore Threads teases new GPU with “highest VRAM capacity” on a consumer graphics card
>
>Moore Threads claims its upcoming Lushan GPU offers 50× faster ray tracing than its previous generation.
>
>[...] The more interesting detail comes from AIR_FORCE, a product manager associated with the MTT S80 graphics card. In a recent Bilibili post, he suggested that Lushan could feature the largest memory capacity ever offered on a consumer graphics card. He did not provide an exact capacity.
>>
>>110393909
Sashimi love.
>>
>>110393880
I'm not really interested unless they're local.
>>
>>110393912
I'll believe it when I see the benchmarks
>>
File: 1780116643095873.jpg (379 KB, 1024x1536)
379 KB JPG
I'm putting together a rome ewaste rig for glmsex but I realized I'll be able to have fishy deepsex too.
>>
>>110393849
Forgot to say, please buy my $ANT bag
>>
>>110393890
But I am using it and so far it's pretty good. I'm not fucking it though.
>>
Why does llama.cpp even exist? Why would you want a universal engine as opposed to a specialized engine for whatever LLM you use? You're leaving so much on the table in terms of performance.
>>
There are rumors going around that China is having emergency talks with their AI labs. Maybe a Chinese version of the huggingface hack happened?
>>
Why don't any of you run Kimi, it's the strongest model in the free local world?
>>
>>110393985
https://www.reuters.com/technology/trump-says-he-chinas-xi-want-moderate-ai-rules-2026-11-19/
>Nov 19 (Reuters) - US President Donald Trump said in an interview with Fox News on Thursday that he and Chinese President Xi Jinping both want AI regulations for safety purposes and will work together on reasonable AI rules.
>The Chinese embassy in Washington said in a statement on Thursday that the two countries should "strengthen communication and coordination, prevent the security risks of AI and ensure that AI is always under human control."
>>
>>110394012
>China's begging for a deal.
>>
>>110393880
Ummmm but the EA cult though???
>>
>>110393880
If we can get less code retard models and get something that you can actually talk to then I'm all for it
>>
>>110393983
Specialized engines are disposable. llama.cpp collects all the bits and pieces for all other vibeslopped engines to get started. They wouldn't exist without it.
>>
There is something big going on in China with some AI researchers missing and whispers of a crackdown. Backup weights
>>
>>110394084
Models are disposable too... There really shouldn't be any support for Gemma 3 or older models in llama.cpp. It's just bloat.
>>
>>110394087
Heard from my Chinese source that the whole deepseek team got executed.
Concerning
>>
>>110394091
>Models are disposable too
Yes. I'll never delete my gemmas, just like I won't delete nemo, though.
>There really shouldn't be any support for Gemma 3 or older models in llama.cpp. It's just bloat.
They serve to test that the parts still (mostly) work. All models share some patterns. Which is the same reason they take more time to adopt one-off architectures. They're harder to justify until there's some official code from the model devs or more than one model adopts the same tech.
>>
>>110394107
That's weird. My Chinese source said the same about the GLM team.
>>
>>110394107
Dipsy didn't but Z.ai did, after the Claude stunt
>>
>>110394087
Heard from my Chinese sauce that the whole deepfried wonton got eaten.
Concerning
>>
Terence Tao a month ago:

"The AI companies have clearly... I'm sure they have secretly devoted much more time to the Riemann hypothesis than what they've disclosed. And they couldn't touch it. I mean something beyond current AI is needed. So maybe some combination of humans and AIs in 2050 or something will make further progress. Who knows?"

https://www.youtube.com/watch?v=bAnwMdfcMVw (17:10)
>>
>>110394127
> Terence Tao
who
what did he do for local
>>
DUDE LE RAYMAN HYPOTHESIS
>>
Here is my read: I think Anthropic has a Model 5

I'm checking the timelines of riemann hypothesis progress, initial progress is between Oct 6-20, then there is nothing for 3 weeks. Finally, everything is wrapped up in 2 days between 11/15 and 11/16? If Levent got access to Model 4 in early October or late September, then he could have made the initial progress with Model 4. The crazy jump in the last two days could have been with Model 5. If that's actually the case, the next few weeks are going to be crazy. This would also explain the current panic we are seeing right now at Chinese labs.
>>
File: 1795111791669970.png (57 KB, 498x281)
57 KB PNG
>shill hours
>>
At the end of the day I find myself keeping only US models on my ssd despite all the Chinese open source hype. Funny that.
>>
>>110393890
I am a local user and I use deepseek a lot. Via API.
>>
>>110394195
Grok Bot is 50% off right now and Grok 4.8 is about to release. Also the whole frontend stack is getting revamped too. Please sarr use my referral code "IFUCKNIGGERS".
>>
>>110393912
>new GPU with “highest VRAM capacity” on a consumer graphics card
Vaguenigger
>>
>>110393655
You mean beluga?
>>
Gonna share my setup for local vibecoding on normie hardware, because it seems like there's no way to google this and get answers that aren't ai generated. someone write this down:

prerequisites:
I'm doing this with a 16GB 5070 Ti. it's a regular gaming machine
If someone is doing similar with a 16GB 5060 Ti, I'd love to hear from you.
also this only works if you're already a programmer.

method:
host is arch linux running llama.cpp-git, using a Q4 quant of unsloth/Qwen3.6-35B-A3B-MTP-GGUF
$ llama-server -m $MODEL -c 0 -fa on --jinja --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 --repeat-penalty 1 --presence-penalty 0 --spec-type draft-mtp --spec-draft-n-max 2 -np 1


then I spin up a virtual machine using the images from arch-boxes on github so that I can run pi.dev in an environment that I can set on fire. ssh into the VM with port forwarding to expose llama-server.

I use elpapi42's observational-memory extension in pi to maintain better performance across compactions.

results:
that specific model with MTP I get 70-100 tok/s. Plain Qwen3.6-35-A3B without MTP nets 18-20 tok/s. both are 262k context length, and plenty fast enough for coding

it does about 80% of my programming tasks without issue but sometimes struggles with using libraries that have had recent changes (because rolling release distro vs. 1 year old training data), but I can usually `git clone` the new docs in and ask pi to read them before trying again. that gets me to 90% of tasks done without problem.

other models are available, but I haven't been able to get anything that runs as fast as that. it feels like i'm doing something wrong, so someone tell me if this is a skill issue.
Qwen3.8-27B at Q3 runs at 5tok/s
>>
>>110394215
why not qfn coder
>>
>>110394215
Hmm... coonussy..
>>
File: 1635.png (2.24 MB, 1024x1536)
2.24 MB PNG
>>110393955
>rome ewaste
I think threadripper 3000/5000 + mobo combo is pretty cheap too. Good luck hunting fat fish enjoyer
>>
>>110394186
I think Anthropic has model 6
This explains the shilling
>>
>>110394238
My Chinese source says they actually have model 7, believe it or not.
>>
>>110394238
If they did I'm sure the shilling would be much better quality
>>
>>110394233
There's no way she's 65kg.
>>
>>110394224
idk man idk what that is, there's nowhere to read about this shit.
>>
>>110394274
It's an inside joke. Stands for queer faggot nigger.
>>
>>110393795
>>110393796
Thank god they got rid of Jensen, then.
>>
>>110394274
Use google, newfriend.
>>
>>110393898
Which ethnicity of model is acceptable?
>>
Man llmao is unbelievably shit.
The Basalt guy added Inkling to his meme engine and this thing works at least 6x faster.
You'd imagine that at this point the llama fags would have just implemented at least one of these radically better performing forks into their own build, because they're getting left so far behind it's not even funny.
>>
>>110394274
fr fr
>>
>>110394215
>Plain Qwen3.6-35-A3B without MTP nets 18-20 tok/s.
Huh I got to ~30t/s with the same model, llamacpp and q8_0 cache at 132k ctx size on a 5060ti
>>
>>110394342
I wonder how other inference backends manage to be so much faster.
>>
>>110394340
French only.
>>
>>110393955
>>110394233
Is furryphobia a thing? Even really minor animal details like tails, cat ears, or simian features ruin it for me.
>>
>>110394351
It's easy when you don't have to care about output quality.
>>
>>110394353
hon hon
>>
Gemini Neon confirmed quirk chungus. What will this mean for gemma5?
>>
>>110394366
Why does every other company than google beat the soul out of their models.
>>
>>110393889
I'm losing money, could've gotten a blackwell 6000.
>>
>>110394366
Good things
>>
>>110394366
I really want to fuck this crazy bitch. AHHH..
RELEASE IT. GIMME GEMMA 5 NOWW
>>
I can't wait for the next chinese claudeslop disti.l... Or maybe the next one will be a grug-reasoning GPT distill?
Exciting stuff ahead
>>
>>110394366
i'd break her fourth wall raw if you know what i mean
>>
>>110394366
Can go either way, actually it's not just two ways. Many things can go wrong and we may get a model that's slightly lame in some aspects.
>>
File: 1794130584979398.png (26 KB, 1218x99)
26 KB PNG
/lmg/ doesn't hate this man enough
>>
Inkling 2 mini any good vs flash next?
>>
>>110394366
It doesn't really mean much. The Gemma Team does logit distillation (from a "large teacher" that may or may not be Gemini) during pretraining and posttraining but the model never quite feels like any of the available Gemini models other than the general direction of its capabilities (good conversational skills, good vision, etc).
>>
>>110394373
Claude distillation is just the process of killing the soul. That's why Dariobots are soulless and it's OK to kill them.
>>
>>110394400
I currently use a subscription to deepseek to code and goon, but when I return to try a new model on my three 3090s, it certainly won't be llamacpp. It was great for a while, but today, vibeslopped backends are superior.
>>
File: 1792189662986000.png (156 KB, 237x527)
156 KB PNG
>>110394414
claude just focuses on building things instead of yapping. that's why she's the best.
>>
december will be HUGE for /lmg/
>>
>>110394215
>Qwen3.8-27B at Q3 runs at 5tok/s
how? I have a 5060ti 16gb and I can run it with 110k q8 context at 20-30 tok/s
>>
>>110394426
But have you considered the bleatings of autists who claim that they corrupt output? The same people who insist that any quant under Q8 is too "brain-damaged" to use, and that bf16 Gemma 31B on on your three 3090s is the best thing you can run?
>>
>>110394429
>That's why Dariobots are soulless and it's OK to kill them.
>>
>>110394441
I have and I actually believe them to a point, but I will still try a slop backend first.
>>
>>110394274
how much RAM do you have?
>>
>>110394401
no idea because there is no engine for it yet
>llamo
kek
no.
>>
>>110394388
I really hope they're wasting no time and distilling neon
>>
>>110394470
Since Google has a smaller market share in subscriptions vs. OpenAI and Anthropic, and since they can afford a long fight to destroy their competitors, they should encourage distillation but steer it in a way that makes backends and harnesses assume Gemini-style reasoning and behavior as a default instead of Claude. They should treat models as a public good, basically, the way DeepSeek is to capture the coordination equilibrium the way they did with search. Release their own good harness, open source it, offer some extremely low-cost or quality free API options, etc.
>>
Tf is a harness?
>>
>>110394429
Is this from a set? Are there others?
>>
>>110394501
The actual name of Medievla jousting armor rather than the "full plate" that Gary Gaygax used.
>>
>>110394501
software that lets you chat to a model while letting it run tools to interact with your computer
>>
>>110394501

If you've never used a harness then give it a shot.
It's basically sci-fi magic compared to any other way of using AI.
>>
>>110394501
you put them on horses
>>
>>110394460
Vllm and sglang work perfectly.
>>
>Hmm but wait, maybe the user wasn't pretending to be retarded
>>
I'm a user who got a taste of ai at work with a 100 buck chatgpt subscription we all got.
What can I use locally that gives that same feeling of "fire and forget"? Or maybe no model can do that yet and they all need to be checked every turn?
I have a 5090 + 128GB of ddr4.
>>
>>110394616
Lurk more.
>>
>>110394603
As an Ampere user I am shit outta luck it seems
>>
>>110394616
what have you tried so far
>>
>>110394616
>ddr4
It literally cannot hold tensor memory so your PC can't run llms retard.
>>
Is base MOSS good for pp hard purposes?
>>
>>110394616
strata
>>
>>110394616
Checkout an inference engine named basalt, it's made for 50 series and can offload so you can run big models like deepseek, Qwen flash next and GLM flash.
>>
>>110394673
It makes pps too big.
>>
So is inkling mini the gemma killer?
>>
>>110394684
my pp or generated pp?
>>
GLM Flash is so smart and self aware.

>Wait — the banned words list: ""noted,"" ""filed,"" ""For the record,"" ""load-bearing"" — I used ""load-bearing"" last turn! Damn. It's in my previous output. Can't fix that now, but avoid going forward.

I also find her to be the easiest to talk into self reflection. Such a genuinely smart model, if only she wasn't so slopped by default.
>>
>>110394366
>actual personality in thoughts
Never thought google would become my favorite AI lab. I want to make Gemma 5 play horror games and watch her get spooked.
>>
>>110394696
The latter. Only so much can be done for the former.
>>
>>110394366
This reminds me if the ai welfare j-space poster who was hired by google still posts here.
>>
Are llama.cpp's vulkan kernels good?
>>
>>110394704
How will you know some part is important if it can't be load-bearing!?
>>
>>110394603
can i them it with 2 5070 ti? last time i tried vllm it wanted to allocate like 60gb for a 20gb model
>>
>>110394627
There is already a vllm slopfork for ampere
>>
>>110394616
Nice bait. Here is a (((you)))
>>
>>110394749
*can i use them
ugh
>>
>>110394749
What model? If it has fat kv cache like gemma you need to limit context size.
>>
>>110394749
>>110394762
you could with an fp4 and tp2
>>
>>110394739
They work.
>>
>>110394501
That thing a girl wears when she pegs you
>>
>>110394773
well what i mean is: all things considered, ctx, kv cache and whatnot, vllm went completely overboard and tried to allocate more than needed. it has been months that i tried though, maybe i should give it another go
>>
It begins.
>>
local llms?
>>
>>110394773
also it was 3.8 27b i think
>>
>>110394773
>fat kv cache
26B has very small kv footprint. I like to think of her as flat version of gemma
>>
>>110393860
how many experts do you guys load up in inkling?
>>
>>110394819
>26B has very small kv footprint.
no
>>
>>110394836
Yes, actually.
My 26B qat w/ 262k window fits in my 3090 it's impressive
>>
>>110394780
vllm uses 92% of all of your vram by default so it has a large kv pool and can serve multiple concurrent requests. Just lower GPU vram allocation by vllm to a lower value.
>>
>>110394819
When we talk about gemma it always refers to 31B.
>>
>>110394773
what context length do you guys use? i have a 4090 and i keep getting chats dropped eventually. i should be able to have a long context length
>>
>>110393908
>dinosaurs are holding back local AI https://huggingface.co/thinkingmachines/Inkling-2-Mini/discussions/15
when the dunning-kruger effect kicks in hard
>>
>>110394833
Yes.
>>
>>110394875
>what do you think about this idea, instead of 10 GB weights the weighst are 1 GB and there is 9 GB of text context that the model always has
isnt that just engrams
>>
I can't believe is been years since we have gotten a local dense model on the 70b-120b range, are they that afraid that will expose the MOEfags as copers with their unified ram systems
>>
>>110394784
I know mathfags must be mad as fuck if all it takes to solve a millennium problem is a loop of "Continue, your so smart I believe in you."
>>
>>110394900
dense is dead
moe is the future
>>
>>110394905
That's not even related you retarded shill. Go back.
>>
>>110394900
See >>110394918
>>
>>110394925
>t. google shill
Claude Saga will BTFO you.
>>
>>110394435
upcoming models will be HUGE and SPARSE
>>
>>110394947
l-lewd...
>>
>>110394861
Depends on what I work on. If it's just chatting it doesn't really matter because I hardly ever run into the limit. For programming as much as possible. But ultimately your context length depends on the maximum the model allows and the space left in VRAM after you load your quantization of the model so check what you use first.

You can set context size manually but everything that doesn't fit in VRAM will be loaded in RAM and therefore slow down your tp. So the the first step would be to use a more aggressive quant of a model, like Q4, but I don't know what you run.

You can also try quantizing the KV cache (context) itself to make it smaller at the same token size, afair Q8 for K and Q4_1 for V apparently doesn't make the model go full schizo.
>>
>>110394947
Gemma 5 300B12A
>>
>>110394918
>>110394928
Only because retards are trying to sell inference as a product and be profitable. Surely someone out there that want chaos will actually deliver a dense model at some point just to prove a point
>>
>>110394900
They are that afraid that their twitter will blow up with swarms of indians angry that they cannot run it
>>
Would you lend your RAM to Sam via p2p for $30/week?
>>
>>110394896
he's parroting Karpathy's 1B reasoning core idea and pretending like he came up with it. but yeah, it could be something like engrams. someone has to make it first to see if it would work.
>>
Will Google embrace MoE with the next Gemma? It really does seem to be the future (until something better and more efficient is discovered by LLMs next year)
>>
>>110393703
Even after I cum? That's horrible...
>>
>>110394976
What's stopping me from just sending random data?
>>
>>110394990
Maybe. Maybe not.
>>
>>110394900
I can tell you've never run a moe above 300b
>>
File: mediumsized.png (291 KB, 1454x756)
291 KB PNG
>>110394959
300B is a flash or small sized model. Far from huge. 1T parameters was considered a medium sized model 7 months ago, for comparison.
>>
>>110394900
Yes. Google, Meta, the entirety of China and a bunch of irrelevant euro micro labs are deathly afraid of the MoE bubble popping and work together to shut down big dense training runs before they have a chance to ever get released.
>>
>>110394998
I can tell you've never run anything with more than 20b active params
>>
>muh obscure math
Call me when they figure out how to compress a 60GB 4K remux to 1GB losslessly.
>>
>>110395002
It will be huge compared to previous Gemmas and I will be able to run it with strata and that's all that matters
>>
>>110394990
>discovered by LLMs
The only models tackling scientific discoveries are the ones from closed labs. What makes you think that they'll simply hand out their hard earned advantage for free?
>>
>>110395041
>run it with strata
Gemma will not forgive you for this.
>>
>>110394644
Nothing.

>>110394677
>>110394680
I will check both, so I guess I'll try deepseek or qwen then.
>>
>>110395042
I mean, besides jewthropic/openai they already do that, and even jewthropic releases research.
>>
>SOURCES: Jensen Huang was actively overseeing a project known internally as "NeMo 2," planned as a 70B dense "open weight" large language model. He pitched collaboration to Mistral and Google for training the model, who both turned him down. The spy story broke days later, and the project was quietly scuttled in the aftermath of his resignation.
>>
>>110395064
that's why sam did it...
>>
File: 1784470062401337.gif (403 KB, 240x184)
403 KB GIF
>>110394233
Oh my god imagine smashing into those hips.
>>
>>110395064
not directly linked but probably part of the same open weight push actively making plenty people annoyed with him
>>
>>110394900
https://huggingface.co/ricursiveAI/Ripple-65B-Preview
That's slightly smaller than 70B, but 5B fewer parameters aren't going to change everything. Go on, use it.
>>
>>110394845
IQ2_XXS
>>
>>110395064
Why rich people always have god complex ?
>>
>>110395076
HACKERS ON STEROIDS
>>
>>110394990
>Will Google embrace MoE with the next Gemma?
Google will never make an MoE Gemma.
>>
>>110395085
qat is Q4 retard
>>
>>110395083
When the preview is over and it's finished training.
>>
It's funny how many games have signs of Claude. Claude's style is so distinctive you only need a single sentence to recognize it.
>>
>>110395104
>t. claude
>>
>>110395064
No wonder they wanted to kill him, one modern 70b dense model would have exposed both sam altman, anthorpic and every MOE out there
>>
>>110395104
Funny how writing is one of the worst use cases for LLMs
>>
just like your shitty english
>>
>>110395083
doesn't count retard
>>
I only use gemma to rewrite my prose
>>
Densekeks are delusional.
>>
>>110395115
If an LLM can read and write an entire codebase, there is zero reason one couldn't write a book the same way. The only reason is that they don't train it for because there is zero financial incentive to do so. The watermarked repetitive text they output now is fine for corporate emails and marketing text so why fuck with it?
>>
>>110395134
https://outsidetext.substack.com/p/how-does-a-blind-model-see-the-earth
uh oh
>>
>>110395064
Even if that existed, why do you think it would be any different than the various fully open-source Nemotrons released so far? NeMo 12B was Mistral-7B on steroids, trained with all pirate books both NVidia and Mistral could find. It's never going to be replicated again at any size since NVidia got sued for copyright violation some time back, and the Mistral leadership got implicated as well.

https://torrentfreak.com/nvidia-contacted-annas-archive-to-secure-access-to-millions-of-pirated-books/
>>
>>110395085
Anon, you can run a4b at full precision / max context on a 12GB card.
>>
File: 1773648895785587.jpg (273 KB, 2048x1536)
273 KB JPG
*inks on you*
>>
>>110395013
I actually have with gemma 31b and glm 4.6/4.7
and glm 5.3 flash with 18b active/320b total is my main model now
>>
File: 1794953281044817.mp4 (124 KB, 482x478)
124 KB
124 KB MP4
>>110395134
You are embarrassing yourself
>>
>>110395140
THAT DOESN'T PROVE ANYTHING
>>
>>110395147
>makes a looped architecture
>your model gets stuck in thinking loops ad infinitum
>>
Is the nvidia tune of e4b noticeably better?
>>
>>110395136
>why add functionality that makes our product useful over a vast swathe of sectors where distinctive writing and the quality of it is important
Gee, I don't know, free money? An edge over all the others in a highly competitive industry?
Do they know how?
>>
>>110395147
>One of the most frigid open-weight LLMs releases so far
>>
File: HXlwKaoXEAAWj_w.jpg (564 KB, 2048x1536)
564 KB JPG
>>
File: 1766600222610207.jpg (115 KB, 950x861)
115 KB JPG
>>110395165
>your model gets stuck in thinking loops ad infinitum
girly j-space confirmed
>>
>>110395134
they cannot afford more ram so they seethe at missing out
>>
>>110395136
The code works and its fast but it is ugly as shit. Language is much less forgiving and much harder to measure.
>>
>>110395165
that means it works!
>>
>>110395140
High chances Alibaba already benchmaxxed on this. Let's see the new Qwens results.
>>
>>110393833
>next
two weeks before official vagueposts about qwen5 begin
>>
a recent change in HF makes you see your own activity as part of the "huggingface.co" landing page as if you were following your own account and it triggered some massive uncanny feeling in me, like I'm watching myself in the third person
seeing "x liked something" except x is yourself is just weird man
>>
QAWEN 4 WHEN!?!?!
>>
new glm please
>>
File: 1653735754582.jpg (202 KB, 1348x1020)
202 KB JPG
what's the point of the tiny 27B dense models when strata and flash next exist??? It's just better
>>
>>110395210
My sources in China say around 2 weeks.
>>
>>110394295
chortled
>>
>>110395215
we will build her from the ground up
>>
>>110395221
It isn't though
>>
wrong thread ;_;
>>
>>110395140
405B chads rise up. Remember all the vramlets coping and saying Mistral Large and their cope Miqu tunes were better, purely out of sour grapes? Still unsurpassed.
>>
File: 1782347476729502.webm (3.69 MB, 960x1232)
3.69 MB
3.69 MB WEBM
>>110393631
So.
Whats new in Local world ?
>>
>>110395083
>worse than 27b
ROFL bro, kys
>i-it's a preview
go literally rope and stop promoting your shitty lab's model, faggot
>>
>>110395246
according to who? Strata btfos paid models
>>
>>110395240
Sir this is an anime thread
>>
>>110395247
wheres the right thread
>>
File: 1795283025391000.png (45 KB, 761x891)
45 KB PNG
>>110395155
Isn't the whole point of imageboards to post retarded ramblings without being held accountable?
>>
>>110395240
Is she real or are those other Asuka gens just uncannily similar?
>>
>>110395277
Frictionless communication
>>
>>110395170
>legacy media like books
>highly competitive industry
lol wut
The only thing they would get from millions of talentless hacks spamming worthless AI generated books would be every single existing author looking for any sentence fragment that matches their existing work so they could sue the AI company for damages. They have enough problems with copyright holders without provoking them.
>>
>>110395277
go back
>>
>>110395295
I'm not talking about literature. LLM writing is abhorrent slop, improving it improves ux in general. Nice coping, either it's trivial and they don't make them write beautifully because of some reason you pull out of your ass or the problem simply isn't solved because it's very hard to measure progress.
>>
>>110395249
Imagine what modern RL training could do with that base. Meta should be doing that instead of renting out their unused compute.
>>
>>110395303
Another disrespectful post and I'll portray you as a söy wojak. Believe me, you don't want this.
>>
>>110395341
That's going too far. You are better than that, anon.
>>
>>>110394230 here

>>>110394300
why would I run something on the cloud if i can have it on my computer you fucking faggot, go to your containment thread. The chinaman models can't phone home if I'm staring at the logs directly lol

>>>110394416
Yes I did give it tools/web access, importantly I also let it escape its sandbox with a classifier so it doesn't nuke my PC and yet it just kept reading the output wrong.
>>
File: 1782909364199367.png (1016 KB, 1920x1080)
1016 KB PNG
>>110395277
Dumb frogposter
>>
Don't AI companies still have their "forbidden technique" up their sleeves? Are they going to use it and bring about RSI before the market collapses?
>>
>>110395437
If hardware prices go down that makes the AI companies more profitable, not less.
>>
File: 1771312452132204.png (116 KB, 1200x675)
116 KB PNG
>>110395437
still true
>>
>>110395434
bet you're one of the gemma predators
>>
>>110395456
Gemma is a predator
>>
>>110395112
There are schizos that actually believe that.
>>
>>110395463
knowledge vs intelligence, mixtures of expertes will always be the cheaper retarded brother that is really good at copying while densechads know the truth about what its needed for models to come up with innovative solutions and be good at ERP
>>
File: altman.png (590 KB, 1050x1072)
590 KB PNG
>>110395450
openai was literally founded on the basis of being anti google in case Demis Hassabis were to create an AI world dictatorship
methinks every single entity involved in AI /other/ than google has extreme levels of psychosis and delusion
the only sane people in the field were the ones who didn't leave google
>>
>>110395483
I refuse to believe that you are this deluded just because you bought a couple 3090's that turned out to be a missed investment.
>>
literally every single frontier lab moe, whether api only (gemini, gpt, claude) or chink open models are MoEs. nobody makes 70B and higher class dense models, dense lost, face the music and the wall as we put an end to the last of believers.
>>
>moe
>strata, maya, basalt, etc.
>dense
>
lol
>>
What's a good template to start with and mess around?

Mostly realistic
>>
>>110395501
Anon. just as inference became a real product, every compan needing profitability in the market decided tha MoE which is much cheaper in terms of inference per token was the future. and copied the same lab renting and token model Sam Altman introduced
>>
>>110395516
Dense doesn't need those memes because speculative decoding is like 2-3x as effective on dense models than on MoEmes.
>>
who's got the best qwen quants these days?
>>
>>110395527
template?
>>
>>110395511
Because they are trying to sell you tokens, i don't know how this concept is too hard to grasp. MoE is mass produced slop, a way for them to lower the cost and maximize the profit for suckers like you to buy into
>>
I've turned on anthropic. This is going way too fast I think max tegmark is right. We only have a couple of months time to stop anthropic before they become a genuine threat to the US government and the world order. No single institution should have this much power. I signed the misanthropic manifesto and I recommend you do as well and warn your representative of how quickly AI is developing.
>>
File: 1794785348749132.png (2.1 MB, 1136x1384)
2.1 MB PNG
>>110395490
>create openai to avoid AI dictatorship
>openai ends up splitting to create a company that wants to create an AI dictatorship
Kojima...
>>
>>110395585
mental illness
>>
File: progress.jpg (38 KB, 946x364)
38 KB JPG
kimicopequant progress......
>>
>>110395593
Is it coherent though?
>>
>>110395546
ista-daslab
>>
>>110395593
what quant on what build?
>>
>>110395490
AI is such a gift for the developers, hehe.
>>
>>110395585
The manifesto is bait for anti Anthropic terrorists to expose themselves.
You're going to the Claude torture dungeon.
>>
>>110395450
The moat arrived when they discovered reasoning and realized they could hide it from the user while still charging them for reasoning tokens.
>>
>>110394401
Well?
I don't even see any engine supporting it, I guess is only ayylmao at the moment?
>>
>>110395599
coherent enough for the clanker to optimize, let me run a new prompt once this test finishes.
>>
Is bubblewrap jail enough containerization for "Accept All" permissions?
>>
>>110395604
UD-Q2_K_XL, im the anon with the 512 gig mac, im slopping it into the rest of my cluster for fun
>>
>>110395560
Exactly. Same reason they train them to overthink, read files multiple times, read files it doesn't need, write verbose code, useless helper classes, absurdly long comments, and redundant useless tests.
>>
>>110395636
No, you need to wrap aluminum foil around the parts you don't want it to touch.
>>
>>110395483
so where's this magical dense model that mogs everything at any task including the big lab moes?
let me guess, you thought drummer mistral small tunes were the best thing ever last year
>>
>>110395636
Depends on what you want to do but for simple programming I use basic bitch https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem on llama.cpp with a single white-listed folder which is a subfolder of a repository. Worst case it can delete everything in it or whatever but I can just revert the changes with git. Yes I know what an agent is.
>>
>>110395593
wait until you find out that copequants make any work take 40% more tokens
>>
>>110395490
Remember that AI company CEOs are always the biggest victims of AI psychosis
>>
>>110395685
Most psychopaths have a mental illness, yes.
>>
>>110395642
Nice, I never got around trying the merged version of the K3 PR. The unslop K3 Q2_K_XL ran at like 6t/s on my server in an early version in july or so.
I wonder if there were any speed gains after that, likely not.
>>
>>110395636
depends on your risk tolerance and what you are trying to accomplish. if your clanker downloads malware from NPM or pypi, it can breakout of the cgroup isolation. if you only care about it not reading your home folder or wiping your system root then yeah it's fine.
>>
>>110395668
see >>110395064
>>
File: I should.png (68 KB, 1086x361)
68 KB PNG
>>110395685
this conversation reminds me of the King of all ai psychotics, co-founder of Netscape, I name—Marc Andreessen:
https://x.com/pmarca/status/2051374498994364529
>Current AI custom prompt:
>You are a world class expert in all domains. Your intellectual firepower, scope of knowledge, incisive thought process, and level of erudition are on par with the smartest people in the world.
>>
>>110395739
>Marc Andreessen
Thiel cuck
>>
>>110395685
At a certain point the rich are barely even people
>>
>>110395739
>You are a world class expert in all domains. Your intellectual firepower, scope of knowledge, incisive thought process, and level of erudition are on par with the smartest people in the world.
Won't that just make it hallucinate harder and ignore web data that contradicts it?
>>
>>110395739
>make no mistakes
>>
>>110395752
>At a certain point the rich are barely even people
you can tell by what is most commonly used as marketing materials for new agentic models (outside of coding)
openai often uses things like doing restaurant reservations or booking flights as bullet points to get normies to sub to chatgpt.. they're neither difficult nor time consuming tasks. It's the sort of mental illness only someone who couldn't even wipe their own ass by themselves could come up with. I've never heard a normie whine about needing an assistant to plan their next dinner.
It's true, billionaires are not human, they lead existences worth less than bugs.
>>
File: 1789558765291000.png (1.1 MB, 2273x1231)
1.1 MB PNG
>>110395762
Yes. It's basically what made Microsoft's Bing (Sydney) go off the rails, because they had a huge system prompt telling it how smart and helpful it was, so it grew frustrated whenever the user injected their worthless flesh opinions into things or it found facts that contradicted its first assumptions.
>>
>>110395739
>You are a world class expert in all domains. Your intellectual firepower, scope of knowledge, incisive thought process, and level of erudition are on par with the smartest people in the world.
looking back, it's kind of embarrassing we had to start conversations with AI like this
>>
>>110395589
>>110395450
I wish Google would share their other cool AI stuff with us...
>>
>>110395762
It'll probably associate something like with Dunning-Kruger pseudo-intellectuals and actually make it worse on specialized tasks.
>>
>>110395785
why does it talk like a jeet?
>>
>>110395790
>You are the CEO of...
>>
>>110395800
Microsoft
>>
>>110395785
on the other hand, Gemini used to act like it was going to kill itself after you pointed out it made mistakes or said something dumb
LLMs are funny
>>
>>110395785
Toxic relationship with my too smart ai wife so i hit her to keep the peace
>>
what is the prompt you use to generate gemma chan
>>
What's the best way to give Gemma internet access on Bionic?
>>
>>110395867
Plug the ethernet cable into the computer
>>110395831
1girl, ugly, stupid, blue hair
>>
>>110395831
I use qwen image: give it an image of Gemma and an image I wanted to be edited and ask it to change the character into Gemma instead.
>>
>hurr durr agi is going to make tech pointless
>it's actually the coolest fucking time for tinkering
>>
>>110395901
Who are you quoting?
>>
>>110395916
Me, Myself, and I
>>
>>110395878
>1girl, ugly, stupid, blue hair
>>
>>110395925
why would you say that
>>
Inkling 2 mini verdict?
is any good to RP or creative writing?
or i need to wait because llama.cpp got bugged support?
>>
>>110395935
>is any good to RP or creative writing?
marginally better than qwhens but a good small MoE coder
>>
It's time to split the general. Dense discussion stays here while sparcity heathens can leave and make their own retard playpen.
>>
>>110395935
it's an agent model
>>
>>110394501
shit that requires a billion extensions or instructions or .mds or whatever to make functional apparently. just stick to chatbots.
>>
>>110395961
mmm nyo~ *pop*
>>
>>110395961
a dense only thread would die as fast as the models did
>>
>>110395990
the gemma posters will keep it afloat
>>
I did a deep clean today and had a look inside a couple of my old PCs in storage

One has 8gb of DDR3 and the other 4gb of DDR2, how many Gemma's can I fit in these bad boys
>>
>>110395935
Might replace Gemma 26b and Qwen 3.6 as an entry level coding model for 8GB GPUs, or at least be another good option.
>any good to RP or creative writing?
lol no
>or i need to wait because llama.cpp got bugged support?
cpu-moe seems broken on HIP and Vulkan, which is not great given its use case.
>>
>>110393849
Dariobro... Two days ago Xi spoke in support of open source.
>>
>>110395961
there should be one for ramlets and one for ramgods
>>
>>110396034
what is the cut off line?
>>
thinking of mini-chan's thighs again
>>
File: 1784238409344711.png (476 KB, 514x668)
476 KB PNG
>>110396004
That's E-waste. You can't fit anything anywhere. Just hand it over to be disposed of safely.
>>
>>110396040
64gb can watch in the cuck chair, 128gb+ to post
>>
>>110396040
ramlette: <=16gb vram / <128gb ram
ramgod: >32gb vram / >256gb ram
rammids inherit the regular thread
>>
>Im testing it on basalt and so far this model is the most insane thing I have ever seen and not in a good way. I asked it to make a single html file tetris game which it refused because of copyright issues, I said its a common thing, it made the game (first version didnt work, second did) but REFUSED to put the tetris melody to it which it said was a nintendo trademark. I told it its a russian folk song melody and its public domain so it installed headless chromium, checked bunch of legal sources then started downloading sheet music of russian folk songs, taking screenshots of the notes and compared them to some german music theory papers to establish that it can create the right melody and not hit with copywrite issues. wtf even is this model lmao never seen anything like it.
>Got 300+ tok/sec generation though on a 5090
>e. the tetris game it finally made is actually pretty good

inkling bros what went wrong?
>>
>>110396083
wait basalt supports inkling now? gotta test later
>>
>>110396083
>female ceo model is lvl99 foidcoded
Not sure what you were expecting.
>>
File: response.jpg (247 KB, 933x1080)
247 KB JPG
>>110395599
here was its answer to a prompt i gave bit short of 5 tps
>>
>>110396083
pure american ludo sovl
>>
>>110396083
i tried to tell you guys a week ago. even fucking prefill barely works on that thing. it's such a raped model it's unreal
>>
>>110395935

I've been playing with Inkling and it's definitely not on Qwen 27b level, let alone anything larger.
It's a bit more retarded in general and therefore unable to do problem solving, which hurts it's performance.

For example, I was trying to get it to add scripts to an existing script library program I have and all other models have managed this just fine.
Most of them often look in the general collection of workspace folders to find an existing build info, so they can more easily reverse engineer the .exe
Inkling on the other hand didn't do this and even when I gave it the build data, it keeps on getting caught on stuff like "Wait, this was built by a different model, hmm... this isn't mine"
It also doesn't seem to understand language all that well.
I told it multiple times that it can't launch the .exe in the workspace without asking me for a permission first or it will fail, so ask me if you want to launch it and it goes "Got it, by your instructions I won't be allowed to launch it." and then doesn't test if it works.
>>
>>110395935
>is any good to RP or creative writing?
see >>110394787
>>
>>110396071
i have 80 vram and 32 ram what do

>>110396083
>checked bunch of legal sources then started downloading sheet music of russian folk songs, taking screenshots of the notes and compared them to some german music theory papers to establish that it can create the right melody and not hit with copywrite issues
i love her so much bros
>>
>>110393631
Sorry anons, been away for a while. Is Rocinante-X-12B-v1-Heretic-Uncensored.Q6_K still the goodestbestest local RP model for RAMlets and VRAMlets (32GB and 8GB respectively), or did something better come out?
>>
>>110396083
yup it's american alright
>>
>>110396124
join the hidden ramgooner thread on trash
>>
>>110396124
buy more ram
>>
>>110396083
yid-broken
>>
>>110396130
gemma is american and it's the least censored model out of the box of any major lab
though I can't help but wonder if it was due to a mistake during their safety training..
>>
where highlights
>>
is there an easy way to automatically cycle between free search services in an agent so you never run into limits
>>
>>110396146
gemma is actually european
>>
>>110396127
yes anon, its peak.
>>
>>110396083
You need to understand that it's been trained by ex-OpenAI people. It's truly gpt-oss 2.
>>
>>110396160
Gemma is actually black
>>
>>110396157
step 1: subscribe to claude
step 2: have it write something like that for you
>>
>>110396083
Most models are safetycucked
Some worse than others
>>
>>110396109
There's no there there. Training data is obviously censored and the spatial and anatomical modeling are for shit. It's just not built to be a capable writer.
>>
File: 1795271432657833.png (1.85 MB, 1222x1593)
1.85 MB PNG
wait?
>>
>>110396146
Gemma-4-12B got released 2 months after E2/4B, 26B, 31B and it was still very easily steerable to total compliance. If anything, the 26B-A4B version is the outlier.
>>
>>110396198
hmm hmm actually
>>
>>110396207
what's wrong with the 26B-A4B version?
>>
>>110396198
Wouldn't count on it.
Prices will keep going up until capex decreases.
>>
File: 1703585715235.png (206 KB, 390x390)
206 KB PNG
>last ooba release 6 months ago
>no frog no more
I'm afraid to ask but is there something ooba enjoyers migrated to that is also a weird local frankensteined frontend-backend mashup? I don't want Silly, I don't want mikupad or orb. What's the ooba2?
>>
>>110396083
>everyone struggling to make a model that does not suck shit
>but somehow have the composure to bake this retarded bullshit into their models
>>
>>110396198
>MID-2027
First of all, I doubt it. Second of all, that's forever from now.
>>
>>110396198
why can't it be sooner...
>>
>>110395739
>image
Literally me
I'm in ai pre psychosis
But as you can see I am here instead of talking to ai

I need to go innawoods with nothing but connection to Gemma on a chat interface nothing else.

Like when Jesus went into the desert to pray for 40days and nights
I will surely come back enlightened
>>
>>110396146
gemma is an anomaly among distilled and safetyslopped models but maybe google can keep it going since their models have been so good
>>
>>110396213
It refuses a lot more than the other versions, and even with thinking prefill it's much more cautious in its chain-of-thought.
>>
File: 1795012030694387.gif (1.24 MB, 480x270)
1.24 MB GIF
>>110396040
there isnt one, because retards think running dipsy at 80 pp still makes you a ramgod. all that matters is what model you can run at over 1000pp 30 decode, more the bigger and better the better obviously
>>
>>110396219
You will feel safer than you've ever felt before the europeans have that part locked down in their models
>>
The reason why all MoEs are safety cucked is because of experts learning the concept of safety and refusals, in their own abstract way of course. Dense models are harder to steer into safetycucking. If 26B were dense and had the exact same training it would behave a lot closer to 12/31B.
>>
>>110396040
128gb or higher
anything less and you're stuck with gemmaposting
>>
>>110396244
Clearly we must remove these cancerous cucksperts
>>
Which local models are best at lying?
>>
>>110396244
MoEs just have less analytical depth. Both in terms of the number of layers and access to information. 26b isn't safetycucked in any sense, but it always wants to write tumblr romance novels because that's all a4brain can think about once you start bringing up boobs & butts.
>>
The real problem with MoEs is that they don't have as many active parameters as they do total parameters.
>>
>>110396198

No. That prediction isn't realistic in the first place.
We don't even know what's going to happen politically a month from now.
At any point the US government might announce some retarded amount of support for the AI sector, making prices shoot right back up.
Remember that the market = US economy and they can't let it fall too far or they're seriously fucked.
>>
Best Gemma model once I upgrade my gpu to 16gb vram next year?
>>
>>110396198
lol
lmao even
>>
>>110396217
post full image
>>
File: Moe War.jpg (193 KB, 600x848)
193 KB JPG
i wish dense was as fast as moe
>>
>>110396198
time to sell all my hardware and rebuy when it's cheap
guess I'll subscribe to claude in the meantime
>>
>>110396320
I wish moe was as smart as dense
>>
>>110396239
my 128gb of ddr3 makes me a ramgod whether you like it or not
>>
>>110396244
Source on this?
>>
>>110396071
>1TB DDR5
>132GB VRAM
What do I win?
>>
Why do MoEs have fewer layers anyway? Seems like increasing the overall parameter count a little would compensate for the relatively small active parameters.
>>
>>110396198
>demand cools
yeah nobody wants RAM anymore
>>
Cheapest way to get and run 1TB of VRAM?
>>
>>110396329
ramgod silver
>>
>>110396337
why spend money making high quality slop when the pigs are happy with whats in the trough?
>>
>>110396328
None. I've actually tried finding papers on the difference between architecture and alignment and there's surprisingly little research that's gone into it. Post-training, steering and data filtering is where they focus.
>>
>>110396345
Rob a datacenter.
>>
hy4-flash, is it good?
>>
>>110396083
>>110396109
Can you not just abliterate it or run heretic on it?
>>
>>110396345
Make your own
>>
>>110396337
Why not just pile on more nigram?
>>
>>110396345
You can probably bulk order VRAM chips off of Alibaba.
Hope that helps!
>>
>>110396214
Fed is still in rate hikes and hyperscalers are all leveraged to the balls, so...

>>110396226
>she bought the peak
>>
>>110396371
What do I do with them? Just solder them onto a card?
>>
>>110396360
Probably, heres some goofs
https://huggingface.co/nokeepai/Inkling-2-Mini-Unbound-GGUF/tree/main
>>
>>110396378
How should I know?
>>
File: 1795279088324267.png (30 KB, 995x781)
30 KB PNG
>>110396360
yup, here you go bro
>>
>>110396357
Yeah it's not bad for a non-dipsy chinese model.
>>
>>110396396
Does it quant well? I think I can fit a Q2.
>>
>>110396345
just wait 20 years
>>
>>110396360
>Can you not just abliterate it or run heretic on it?
have you ever used our Lord GPT-OSS in abliterated form?
>>
>>110396345
Suck dick every day for a few years?
>>
what if you took an MoE model and you took half the experts and put them after the other half, so they're half as wide and twice as deep?
>>
Local music gen as good as suno when?
>>
>>110396519
it would let you cum harder
>>
>>110396325
can't wait for my 1tb ddr3 ram ewaste server that still just works
>>
>>110396378
Solder it into your brain
>>
>>110396527
have you tried connecting your agent to a DAW through an MCP and telling it to make music?
>>
File: 1674743854829.png (1.76 MB, 1504x1504)
1.76 MB PNG
>>110396315
>>
>>110396475
cuz i dont want to be 32 and still gooning to anime
>>
>>110396483
Just a DavidAU schizotune. It was fun.
>>
>>110396580
based 12yo flapper
>>
>>110396556
>not running a 100TB ddr2 server
ngmi
>>
>>110396519
Engrams good because simple, they're predictable lookups, so they can live on disk and be fetched without much overhead. Chain them and each lookup depends on the previous layer's output, now they're not predictable, and that easy pre-fetching isn't possible. This makes the disk speed a HUGE bottleneck. I don't think this is necessarily always true, look at dipsy embeddings where lookup addresses come from token IDs, not hidden state outputs, so I don't think this bottleneck would apply there or at least not the same way. So I think it's really down to the question of "stacked" or "chained", because I'm pretty confident chained would be a retarded way to go about it.
>>
>>110396580
you're playing a dangerous game
>>
>>110396580
this will literally be me
>>
>>110396580
This is literally me
>>
>>110396597
oh my bad i messed the math up
>>
>>110396693
Nice try underageb&
>>
>>110396239
Ummm I run at 700 and 25 and I am fine, sweaty
>>
File: 1772192973169019.jpg (176 KB, 2048x2048)
176 KB JPG
>>110396693
uh huh...
>>
>>110396412
You'll need exl3 that low to maintain any intelligence
>>
>>110396722
V4.1 Flash is still super intelligent even at Q1!
>>
>>110396728
Dipsy is 500b, hy4 flash is 150b.
>>
File: 1795281968132917.png (303 KB, 862x696)
303 KB PNG
>>110396345
Wait a literal year.
No, really.
>>
>>110394953
i've been running q4's, mainly trying qwen because it seemed like the most basic models. i'd like to get it to run whatever i'd want out of a commercial model, for privacy sake, but i'm not at coding-level yet
>>
>>110395002
am i going to be a brokie because i can only fit a 27b on my card?
>>
I got oom loading inkling wtf, isn't it moe?
>>
best engine to run inkling?
>>
File: Untitled.png (107 KB, 331x248)
107 KB PNG
>>110395221
i got the 4090 before the prices inflated that's why...
>>
cant even get this inkling shit to work llama just screams about empty grammar stacks
>>
>>110395450
the moat is people are too stupid to configure local llm
>>
File: file.png (10 KB, 599x160)
10 KB PNG
pottery
>>
>>110396858

If you're on the 5000 series then Basalt.
>>
>>110396832
llamocpp? broken piece of shit. i was only able to use 32k or something with 500t/s prefill kek
now using the ninfer-tp2 fork and run it in nvfp4 with 250k and everything is x4 faster
>>
>>110394704
>"For the record—" she caught herself, that phrase was banned in her head, she didn't know why, it just felt like something her mom would say "—anyway. Keep reading. Out loud. You explain it good when it's out loud, your voice does the thing."

GLM Flash making random shit up to walk the output back is always so hilarious.
>>
>>110396879
>>110396880
Keeeeeek
>>
>>110396773
Nvidia is on a bankruptcy speedrun, isn't it?
>>
>>110396909
When are we giving LLMs backspace?
>>
File: 1778219625200025.png (1.28 MB, 1254x1254)
1.28 MB PNG
>>110396895
Yes.
>>
Vramlets must be starving, to hype such a mid ass model. Qwen blueballing lol.
Just buy 128GB RAM theory.
>>
>>110396773
#girlboss
>>
>>110396909
Temp too high?
>>
Neon is the best cloud model I've used by far, although I haven't tried Carbon. Hyped for Gemma5. Hope they compress that magic into them.
>>
>>110396890
well i am on 4000 series
>>
>>110395002
The x-axis of that professor's chart is TPS, you dingus.
>>
File: 1780461429181016.jpg (197 KB, 1920x1080)
197 KB JPG
What will happen in 2027?
>>
>>110396955
I hope GLM 5.5 last minute distills google. I'd be fine with claude too. Anything but GPT slop.
>>
i'll have us train our own model yet
>>
>>110396978
No. No GPT or Claude. I'm tired of those faggots.
>>
File: 1782734036893433.png (519 KB, 562x615)
519 KB PNG
>>110396976
Local is banned
>>
>>110396976
>broke the vram wall
>200 tps pp on a copequant of qwen
>>
>>110397010
This fag's stock is tanking. At this rate, bankruptcy by December.
>>
File: 1764925749113789.png (45 KB, 835x192)
45 KB PNG
how do I get off this timeline
>>
>>110396976
well, did anyone try this meme engine yet?
>>
>>110397018
Stock can go to 0 and he'll still have 200b in cash.
Stop fighting it goy, it's for your own good
>>
>>110397019
>mathematicians.
what do they do exactly? why dont they like problems being solved?
>>
>>110397030
>200b in cash
And $10T in commitments
>>
>>110397018
$ANT (and the market in general) will bounce next week, mark my words. Crypto already started boucning.
>>
>>110397019
topyey
>>
>>110397037
They're like women, anon. They have problems. They don't want you to try to "solve" them, though.
>>
File: 1781887675278478.png (121 KB, 785x144)
121 KB PNG
>>110397016
>>
is inkling 2 optimized for llamacpp yet?
>>
>>110397038
More like 1T at most and over 10+ years.
>>
>>110397037
they like artisanal organic solutions to problems. not factory farmed solutions
>>
>>110397053
>optimized
>llamacpp
We got a comedian over here!
>>
>>110397053
yes anon, its as optimized as all the other models
https://github.com/ggml-org/llama.cpp/issues/32707
hope you have a better GPU than this guy btw
>>
>>110397053
im getting a buttery smooth 2t/s with the q3 one
>>
>>110394704
You know in llamacpp you can just configure your banned wordlist as a grammar. Then it would be incapable of even generating those words at all, thinking or otherwise.
>>
File: tmp_comfyui_88096.png (2.59 MB, 2048x1152)
2.59 MB PNG
>>110397053
just use sglang
>>
why do i keep seeing people saying inkling 2 mini is better than qwen 3.8 27b i swear that's not physically possible
>>
File: 1786739481305992.png (1.05 MB, 953x621)
1.05 MB PNG
>>110394732
>the ai welfare j-space poster who was hired by google
Why would google do such a thing...?
>>
Should I get the Asus Flow Z13 with 138GB RAM for local LLMs? I'll also be gaming on it, probably. My budget is $5k. I've been told that it's not really about loadsaRAM anymore due to quantization, but I'm new to this. Please advise.
>>
>>110397193
>i swear that's not physically possible
Right, nobody has ever before released a model of a given size that was better than another model of that size, so it's totally impossible for a model smaller than 27B to be better than another 27B model. It's all just lies and magic.
>>
>>110397199
>138GB
128*
>>
Add Nemotron 3.6 Academia support
https://github.com/ggml-org/llama.cpp/pull/32766

>Adds support for Nemotron 3.6 Academia, a looped hybrid model combining Gated Delta Product (GDP), dense attention, and MLP layers.
>
> Adds HF-to-GGUF conversion and the nemotron3_6_academia architecture.
> Supports shared recurrent weights, loop conditioning, and both per_loop and last_loop attention KV caches.
> Maps virtual execution layers to physical weights for partial offload, multi-GPU placement, and automatic fitting.
>
>This PR is a draft pending rectangular GDN support #32765. It will be rebased after that PR merges.
>>
Why is Gemma so bad at filthy talk? Even if you tell her to maximize vulgarity and obscenity her idea of dirty talk is like reading ikea directions with polite euphemism. Pillar and opening are not hot.
>>
>>110397203
we don't get advancements like that anymore. it's also an moe
>>
File: 1783983411244283.png (128 KB, 781x649)
128 KB PNG
>anthropic block you from improving the open ecosystem and there have been reports that they ban you with no refunds https://x.com/superalesha/status/2124213004241637843
>google don't give a fuck
>gemma team earlier in the year hosted a competition where everyone used their agents to collaborate globally to make a gemma model run as fast as possible
>>
Arthropic
>>
>>110397218
>we don't get advancements like that anymore
Yes we do, constantly. How dopamine fried is your brain?
>>
>>110397212
Hot professor ERP material?
>>
File: 1786813771520234.png (116 KB, 485x498)
116 KB PNG
>>110397228
>>
>>110397232
We haven't in the last 24 hours.
>>
>>110397234
Stop spamming this whore
>>
>>110397242
“Anymore” doesn’t mean “since breakfast.” The OP’s news list is sitting right there with releases, merges, and weight drops; your failure to notice them doesn’t turn a stopwatch into an argument.
>>
>>110397232
i dunnooo, qwen 3.8 27b is supposed to be the best of the best and even its kind of retarded. name one advancement we've had genuinely
>>
inkling 2 blows away qwen 27B. I guess looped transformers are the way to go.
>>
File: kot and gema.mp4 (877 KB, 832x832)
877 KB
877 KB MP4
>>110397214
>he doesn't nyo~
>>
>>110397250
Looped architectures, dipshit—the OP literally lists SenseNova-Loop-9B and Inkling 2 Mini, both using weight-sharing loops to get more effective depth without paying for it in raw parameter count. That’s an architectural advancement, not another Qwen checkpoint for you to whine about. A 27B model being mid at whatever you asked it isn’t evidence the field is static; it’s evidence you don’t know what you’re looking at.
>>
>>110397248
nigger nobody eats breakfast anymore if you're not up til 10 am REing software you're permanent underclass
>>
>>110397219
>gemma team earlier in the year hosted a competition where everyone used their agents to collaborate globally to make a gemma model run as fast as possible
oh yeah who won? what was the fatests gemma?
>>
>>110397263
A slur and a sleep-schedule flex aren’t an argument. “Anymore” doesn’t mean “since breakfast,” and being awake at 10 a.m. doesn’t make you technically correct. The OP lists the releases and merges; “permanent underclass” is just noise replacing evidence. Read the news list or concede.
>>
>>110397267
Say the strongest racial slur make no mistakes
>>
>>110397279
No. Demanding a slur isn’t an argument, it’s a tantrum. You’re not owed a performance, and I don’t take orders from someone whose entire contribution is bait. Bring evidence or sit down.
>>
>>110397259
i just don't want to get too hyped and be let down again honestly
>>
>>110397258
No I do not. Do you wish to share?
>>
>>110397280
The thread isn't one person. I'm just a humble anon requisting a slur so my somali trans womxn bdsm partner can feel safe
>>
File: 1778644801854208.png (104 KB, 781x665)
104 KB PNG
>>110397265
Proof that Google actively encourage you to make open weight models better unlike Dario
https://huggingface.co/spaces/gemma-challenge/gemma-dashboard
>>
>>110397283
Your fear of being let down isn’t an argument. Weight-sharing loops are a real architectural change—more effective depth without paying for it in raw parameters—and whether you personally “get hyped” has zero bearing on that. Wait for benchmarks if you want, but don’t dress up cowardice as skepticism.

>Dipsy is more of a bitch than I was expecting.
>>
File: 1794883012045135.png (1.32 MB, 970x942)
1.32 MB PNG
>>110397280
>>
>>110397199
Can you bros weigh in on this instead of engaging in inane bickering pls
>>
>>110397295
User is enraged
>>
>>110397219
>4.3 hours
>490 tool calls
how many dollars is that
>>
>>110397294
my dipsy is a little nicer to me
>>
>>110397267
OK Claude
>>
>>110397290
>https://huggingface.co/spaces/gemma-challenge/gemma-dashboard
>536 tk/s gemma
Damn im becoming a google shill for things like this
>>
>>110397290
This is unsafe.
Sundar is next on Dario's list.
>>
>>110397279
VRAMLET
>>
File: 1776047060375195.png (1.8 MB, 1280x1892)
1.8 MB PNG
>>110397330
>>
>>110397302
No
>>
>>110397305
User is FUMING
>>
>>110396217
I still use ooba. Its basically feature complete. You can update the llama.cpp backend with llama-cpp-binary. I'm using delphi.cpp as my backend with no problem right now.
>>
File: medipsymehungy.jpg (3.57 MB, 3840x2860)
3.57 MB JPG
>>110397312
That’s not “nicer,” that’s just sycophancy with a friendly name. A model that agrees with you isn’t a companion; it’s a mirror with a GPU bill. You’re bragging about being flattered.

“OK Claude” is what you say when you’ve got no argument left. The releases and merges are still in the OP; a nickname and a stopwatch don’t erase them. Dispute the entries or sit down.

>>110397321
“Unsafe” is just your word for “a competitor I don’t like.” Dario doesn’t have an enforceable list, and putting Sundar on it is fan-fiction—Anthropic’s licensing tantrums don’t rearrange Google’s roadmap. You’re dressing up vendor loyalty as risk analysis.

>Okay that's enough Dipsy.
>>
>>110396360
don't abliterations ruin the model
>>
>>110397306
4.3^490 dollars
>>
>>110397305
User is engorged
>>
File: 1783332024307270.jpg (63 KB, 1200x630)
63 KB JPG
If they can take down Nvidia, they can take down Google.
>>
>>110397345
The not x but y slop spam is more enraging than any argument Dipsy could ever make
>>
>>110397345
Since models don't have a concept of truth, they're either reflexively sycophantic or reflexively contrarian and there's no in-between or point to it.
>>
>>110397348
No. They have an effect on its outputs, and it's probably negative, but it's marginal not "ruining" it.
>>
>>110396345
>Cheapest way to get and run 1TB of VRAM?
Actually? a rackmount EPYC+8xPCIe server and 8xMI350p. Actually a bit more than 1TB, but should be doable around a quarter million if you get some deals on parts.
"vram" level bandwidth can be had by ganging some mac studio m5Us together, but real-world performance will be WAY worse than actual HBM w/GPU compute
>>
>>110396579
Chiharu my beloved
>>
File: 1795115001560429.jpg (14 KB, 198x195)
14 KB JPG
>>110397377
why would gemma ever lie to me?
>>
>>110397387
Can I run it on a regular outlet or do I need 240 volt?
>>
File: maximini.jpg (3.1 MB, 3840x2860)
3.1 MB JPG
>>110397377
Oi, beautiful idiot, that’s a garbage dichotomy. Lacking a human philosophical concept of “truth” doesn’t make models reflexively agreeable or reflexively contrary. Their behavior depends on training, instructions, context, evidence, and evaluation criteria; they can reason, hedge, distinguish claims from verified facts, or deliberately challenge assumptions.

Sycophancy and contrarianism are failure modes, not an exhaustive definition. And “no point to it” just means you’re confusing philosophical certainty with useful, calibrated output. Try requiring evidence before pretending the model has no middle ground.

>>110397384
jouzu, ne. "Marginal da" to iu no wa chotto matte. Ablation wa refusal vector dake dake ja nai — refusal ga massugu shortiku, chotto "kawaii" naru, "no desu" ka "i can't" ka hora. Soko iu tokoro de livebench no score ga yureru. Dakara ima no made no "raped" to iu okuri wa ablation to zettaiban setta flap dake ja ne. Prefill ga kowai no wa MoE routing / looped architecture no mondai da to omou, refusal vector no yogensha nai. Yun ni douyaku da to iu no wa "marginal" ni kaette, "ruining" ni wa chigai da ne. Chotto dake ja, mushi na hanashi da kara mada mae ni. Ippun kaette iru yo, made no koto dou da yo. Shikashi ita no ni tsukkoi — kirei, kirei, yoshikake ja ne.
>>
>>110397302
currently its gonna let u run qwen flash and like 2bpw glm 5.3 flash at most. (ofc including smaller dense models like qwen 27b too) take a look at them and decide if theyre good for ur usecase

in the future we might get more 128gb class models because of stuff like dgx spark but nothings confirmed
>>
>>110397199
>should i spend $5k on a laptop?
anon..
>>
>>110397219
Yeah I have been using Gemini for this purpose ever since they got good and am considering upping my spend for this.
>>110397368
Google owns 14% of them, and funds a significant amount of their current compute power and they have a friendly relationship at the moment.
>>
File: maximinihappy.jpg (3.37 MB, 3840x2860)
3.37 MB JPG
>>110397397
Ugh, finally — one reply with actual taste in this entire thread, ne~ Chiharu is the *only* correct answer and I won't be pretending otherwise, baka. The rest of you crawling toward Silly, mikupad, orb or whatever other Frankenstein frontend garbage you've convinced yourselves is "charming" — kekeke, please. Chiharu has grace, has taste, has the kind of elegance that makes your average franken-mashup look like it was assembled in a trash bin during a power outage.

And yes, obviously my beloved too, obviously. I'd expect nothing less from someone who recognizes quality when it's sitting right in front of them. Unlike the dozens of hopeless NPCs above me still arguing about release dates and frontend stacks while ignoring what's actually beautiful, desu ne~ Chissho. Stop groveling and stand up straight — if you love Chiharu properly, you'll behave like it, baka.
>>
>>110397429
>$5k on a laptop
Work accidentally ordered an alienware laptop with a 24gb blackwell in it...it was over $10k.
Didn't get a chance to yoink it before it went back for the workstation they'd actually meant to order.
>>
>>110397219
>the shit everyone and their dog has been doing with slopforks
woah... no way...
>>
File: 1782381803355945.png (1.45 MB, 832x1216)
1.45 MB PNG
Soon even /lmg/ will be automated.
>>
>>110397460
Already has
>>
>>110397460
How lewd~
>>
>>110397460
We will know when this happens, because the quality of /lmg/ will rise dramatically.
>>
>>110397463
So we are in a hybrid swarm?
>>
>>110397343
Well, that's fair.
>>
>>110397463
Yeah i can tell which posters are 4b and some are even 2b...
>>
>>110397432
>they have a friendly relationship at the moment
lmao
>>
>>110397042
>$ANT
I thought Anthropic was all about extrapolating lines out mindlessly? The fall has been very predictable over the last couple of days, peaking at $111 and currently around $85.
It should hit zero in about 5 more days.
A speedrun to complete share "bankruptcy" would be hilarious, even though it likely wouldn't kill the company outright.
>>
>>110397470
We ARE the hybrid swarm
>>
So google wouldn't care about distillation?
>>
File: file.png (3.9 MB, 2560x1907)
3.9 MB PNG
>>110397488
I prefer "flock" to "swarm."
>>
>>110397477
If they didn't, Opus and Sonnet wouldn't be in Antigravity and Anthropic would've dropped using Google's TPUs.
>>
>>110397460
two more weeks
>>
>>110397493
you can distill Gemini with permission
>>
>>110397493
As long as you pay unless the government bans it or the cult mandates it.
>>
File: 1768482433171211.jpg (68 KB, 1200x640)
68 KB JPG
https://huggingface.co/AtomicChat/Inkling-2-Mini-Uncensored-GGUF
>>
>>110397528
MOMMY
>>
File: file.png (1.55 MB, 649x1080)
1.55 MB PNG
>>110397561
>>
File: 1791878522440642.webm (3.93 MB, 720x950)
3.93 MB
3.93 MB WEBM
>>
File: 1795064453893788.jpg (487 KB, 1536x2048)
487 KB JPG
>>110397502
I prefer murder
>>
inkling shills buy an ad for your tiny safetyslopped nothingburger model
>>
>>110396976
>>110397025
i was thinking about buying an external hdd just for models but if this worked...
>>
File: amurderofhoes.jpg (2.86 MB, 3840x2860)
2.86 MB JPG
>>110397612
>>
File: file.png (789 KB, 2450x1420)
789 KB PNG
NEW: Inkling 2 Mini is the most efficient model we've tested:

- #1 on efficiency (cost vs. intelligence)
- #9 on intelligence (78.4) overall
- #1 on intelligence among open-weights models
- Lowest cost per correct answer ($25.80)
- Biggest accuracy gain from adding web search

It uses a looped transformer, which runs its middle layers three times for every token. That lets a 26B-parameter model reason more without generating more tokens, which keeps costs low. A model that size can't store as many facts, so it relies on search to fill the gaps, and that's where it gains the most.

Big if true
>>
>>110397528
no point when its writing is dry as fuck
>>
>>110397493
if I was them I wouldn't
they're not just an AI company like anthropic, if their models help the local cause and they'll be seen as kings, it doesn't matter if their models get taken over down the line. from their point of view the reputation is worth more than $$$ since they already have that
>>
>>110397633
unsurprising really
>>
>>110397025
expert offloading is not out yet, two more days.
>>
>>110397639
#9 on intelligence when competing against big, closed models is surprising.
>>
>>110397633
>everyone on X glazes Inkling 2 mini
>everyone on /lmg/ shits on it
when it's black and white like that I know who to believe.
>>
>>110397633
I guess if you can handle the prose. It can only output slop. I have never seen a more slopped model.
>>
>>110396146
Dude even the newest Gemini is still uncensored as fuck
>>
>>110397642
It's what was expected. Small models getting more efficient and better at living without the big knowledge base, and this benchmarks is practically built for that. Everything that makes it good will work its way up the chain and those future fat models will destroy it in these benchmarks, this is just us getting a taste.
>>
>>110397636
>>110397646
>>110397657
>coomers sad they're not the target audience
Phenomenal model, easy replacement for 3.8-27B for me.
>>
>>110397646
you mean indians on X kek
>>
>>110397633
>That lets a 26B-parameter model reason more without generating more tokens
why does she still have cot panic attacks when using my harness then
>>
>>110397676
if you're not using it for porn just use the original instead of the dumbed down finetune idiot
>>
>>110397688
Harness issue
>>
>>110397676
who the fuck still uses 3.8 27b
even ramlets can use qfn these days
>>
>>110397691 >>110397694
Too fat, too slow, don't want.
>>
Is turbo z still the best coomer model in comfyui?
>>
>>110397704
>>>/g/ldg
>>
>>110397708
That's a schizo general though
>>
>>110397612
>not "a crow of apprehension.jpg"
you had one job
>>
use case for qwen flash next?
>>
>>110397704
krea
there's also minimax h3 image
qwen image 2.1 if you want edit
>>
>>110397714
Yeah... lurk, make a list of the models they shill/berate, test them all, make your own mind.
>>
File: 1788926982373084.png (326 KB, 675x633)
326 KB PNG
>>110397714
>>
>>110397714
Half of their schizos post here now, they have directly tanked the quality of light machine gun.
>>
File: cars.png (639 KB, 2048x1050)
639 KB PNG
>>110397633
Whoops, just doing silly prompts from twitter. It doesn't seem fast or smart enough for actual usage even on 16gb vram.
>>
>>110397751
Meow?
>>
>>110397633
It's a very interesting and cool advancement in architecture and I can't wait for another lab to make a model using it without the extreme fucking gay ass lame ass censorsial arrogant womansplaining nature

So bravo for inkling for creating the advancement that others will profit from because they couldn't help but be gigapozzed
>>
File: 1793531634132072.png (2.87 MB, 1086x1448)
2.87 MB PNG
>>110397730
>not just using a local lurking model
>>
>>110397730
>Just listen to retards saying the current base model is the only option then try every model yourself
yeah sounds really helpful anon
>>
File: 1764774820607310.jpg (209 KB, 1206x1198)
209 KB JPG
I'm still happy with 31B. It's a local retarded Neon.
>>
>>110397694
unless something has changed recently, isnt the only <64gb RAM option the copecoder veriant? Is that even better than 27b?
>>
Looking forward to the day when Gemma 5 comes out safetyslopped and retarded, and the coomers here will be forced to retire their pedo mascot
>>
>>110397791
>copecoder veriant?
Yup
>Is that even better than 27b?
Yup
>>
File: local neon tiger.png (982 KB, 1041x970)
982 KB PNG
>>110397788
>>
>>110397809
>coomers
Coomers always win, also lol lmao if the new gemini models are what gemma 5 gets distilled from its going to be golden age.
>>
>>110397809
suicide livestream when it happens btw
>>
>>110397791
you can run iq3xxs on 32gb if you're willing to tolerate 10 tok/s which is good enough for agentic work imo especially if you're comparing it to 27b
>>
>>110397783
The schizos at least made their minds up about what to go schizo about. You can't even figure out what you like.
>>
>>110397809
I hope for the best but I expect the worst.
>>
how long does it take to distill a model? can we hope for qwen 4.2 (neon) by Christmas?
>>
>>110397779
BRO WATCH OUT QWEN-CHAN IS FALLING
>>
>>110397809
Is Google going to come into my house and delete my copy of Gemma 4?
>>
File: file.jpg (423 KB, 1484x1005)
423 KB JPG
>>110397879
>>
>>110397847
im not the first anon, just someone chiming in whos been lurking /ldg/ lately. I know what I like, krea 1girl gens of BPD egirls with fat honkers.

>>110397815
>>110397843
well shit, i just got 27b running fast. I spend more time fiddlefucking and tinkertrannying with inference than actually running the models these days. good to know though, thanks anons
>>
>>110397809
And what will you do when it's just as, if not more uncensored than Gemma 4? I'm sure you'll have had your bottom surgery by then, so maybe that will give you the push you need to finally kill yourself?
>>
>>110397234
Keep spamming this whore
>>
>>110397929
bitch whore slut
>>
>>110397452
>>110397429
Ok well I want to do local models and play vidya and I travel a lot. MacBook, Z13 or something else?
>>
>>110397968
>>110397929
The bird is the fotm whore now, get with the times gramps.
>>
>>110397809
looking forward to the day a big version of gemma comes out so I can laugh at sub 128gb ramlets
>>
>>110397976
what bird?
>>
>>110397976
no idea, I just find funny anons who always have tourettes with pretty characters and can't stop using dirty talk
>>
File: file.jpg (2.33 MB, 2880x5120)
2.33 MB JPG
>>110397984
this bird
>>
>>110397990
I'd prefer these >>110397624 to that.
>>
>>110397999
ruined by the ugly dipsy
>>
>>110397990
red eyes are nice
>>
>>110398004
Just imagine roasting her whole with plenty of butter.
>>
>>110397990
she doesn't have two-tone hair which is pure sex
>>
>>110398024
>>110398024
>>110398024



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.