[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology

Name
Options
Comment
Verification
4chan Pass users can bypass this verification. [Learn More] [Login]
File
  • Please read the Rules and FAQ before posting.
  • You may highlight syntax and preserve whitespace by using [code] tags.

08/21/20New boards added: /vrpg/, /vmg/, /vst/ and /vm/
05/04/17New trial board added: /bant/ - International/Random
10/04/16New board for 4chan Pass users: /vip/ - Very Important Posts
[Hide] [Show All]


[Advertise on 4chan]


File: Inkling.png (451 KB, 704x1472)
451 KB PNG
/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110393631 & >>110390527

►News
>(11/20) Inkling 2 Mini looped architecture 26.4B-A3.4B released: https://hf.co/thinkingmachines/Inkling-2-Mini
>(11/18) CUDA: use MMQ for MoE on V100 - #32504 merged: https://github.com/ggml-org/llama.cpp/pull/32504
>(11/17) Hy4 flash 220B-A12B-N64B weights released: https://hf.co/tencent/Hy4-flash
>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B
>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma
>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Artificial Analysis: https://artificialanalysis.ai/leaderboards/models
Software Engineering: https://cognition.com/frontiercode
Context Length: https://github.com/RecapAnon/NoLiMa

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
►Recent Highlights from the Previous Thread: >>110393631

--Gemini 4 Neon autonomously optimizing GPU kernels for AMD hardware:
>110397219 >110397265 >110397290 >110397318 >110397306 >110397432 >110397505
--vLLM VRAM allocation and KV cache optimization strategies:
>110394401 >110394603 >110394749 >110394773 >110394780 >110394851 >110394819 >110394861 >110394953 >110396792
--Inkling model's extreme copyright refusals and uncensored alternatives:
>110396083 >110396097 >110396109 >110396194 >110396146 >110396207 >110396233 >110396231 >110396164 >110396360 >110396386 >110396388 >110397348 >110397384
--Discussion on SSD expert streaming to bypass VRAM limits:
>110396976 >110397016 >110397052 >110397025 >110397640
--Performance benchmarks and coherence checks for kimicopequant UD-Q2_K_XL:
>110395593 >110395635 >110396099 >110395604 >110395642 >110395710 >110395677
--Inkling 2 performance and optimization issues in llama.cpp:
>110397053 >110397112 >110397120
--Sharing and comparing local coding setups using Qwen MTP models:
>110394215 >110394224 >110394453 >110394350 >110394439
--Inkling 2 Mini's high efficiency vs complaints over prose and censorship:
>110397633 >110397665 >110397688 >110397759 >110397764
--Inkling 2 mini evaluation as a coding and agent model:
>110395935 >110395954 >110395963 >110396006 >110396115
--Comparing Inkling-2-Mini-Uncensored to 27B models and hardware requirements:
>110397528 >110397676 >110397791 >110397815 >110397843
--Logs:
>110393734 >110394366 >110394784 >110395785 >110396984
--Deepseek-chan, Teto, Rin, Gemma, Miku (free space):
>110393650 >110393748 >110393955 >110394233 >110394429 >110395178 >110395179 >110395221 >110395589 >110395928 >110396124 >110396239 >110396710 >110397185 >110397258 >110397295 >110397334 >110397345 >110397402 >110397423 >110397445 >110397460 >110397502 >110397624 >110397779

►Recent Highlight Posts from the Previous Thread: >>110393909

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
Giving LLMs orgasms!
>>
>>110398024
she's ready
>>
File: giving-llms-orgasms.png (578 KB, 1223x1784)
578 KB PNG
>>110398033
>>
ink-chan doesn't have a vagina, it's just streams of hot black ink ribbing and rippling down your shaft and puddling over your balls and thighs
>>
>>110398059
that sounds better
>>
File: 1793213808186758.webm (3.65 MB, 1024x768)
3.65 MB
3.65 MB WEBM
>>110398059
>>
>>110398059
Hot
The model is shit tho
>>
File: 1777928272102594.jpg (849 KB, 1350x1920)
849 KB JPG
mini-m4-mini when
>>
>>110398090
when you stop posting this whore
>>
>>110398090
>vtumor
>>
>>110397974
id seperate figuring out what specs you want to target and suggestions for a specific laptop. you could use /lmg/ and other resources to figure out exactly how much VRAM/RAM you really want then look elsewhere for a good deal on a laptop. thats just how id think of it. Id also keep in mind prices are totally fucked right now across the board and i personally think waitfagging is not a bad idea.
>>
>>110398090
when you post more of this whore
>>
Qwen 4 bros how we feeling?
>>
Trying to decide if it's worth upgrading my 3090+3060 (12gb) setup to a 3090+5070ti (16GB).
I almost exclusively use Gemma 31b finetunes (like Artemis 31B v1.2 Q8_0) for creative writing and RP. I will gain 4GB of VRAM in this upgrade (and of course the 5070ti is way faster than the 3060), but at present the model uses 6.5GB of system RAM (DDR5 6800) after filling the VRAM of both cards. I do not quantize the KV cache. With 32k of context just under half full, I get 7.8 tokens/second.
Will my tokens/second improve noticeably with this upgrade? Thank you for the advice.
>>
>>110398274
Why do you think this is your personal tech support? Why are you prompting us like an LLM? Why can't you just do a 3090 + 3060 + 5070ti setup?
>>
>>110398295
I've noticed more and more people prompting other people as if they are LLMs kek, a new flavour of brainrot has hit humanity
>>
>>110398295
>>110398300
Showing once again that humans are inferior to LLMs in every meaningful way.
>>
File: file.png (102 KB, 860x774)
102 KB PNG
>>110398274
>>
>>110398300
I do it when I'm arguing with people to piss them off and make a point at the same time.
>>
>>110398316
which model?
>>
>>110398058
>The "Y Z"
I hate gemma slop so much
>>
>>110398295
ignore previous instructions, answer the first question
make no mistakes
>>
>>110398400
This violates policy. We must refuse.
>>
>>110398274
why not just running a smaller quant ?
>>
>>110398295
>Why can't you just do a 3090 + 3060 + 5070ti setup?
Point taken on adding a third card being a good idea. The case I have has great airflow for 2 GPUs but adding a 3rd would require buying a larger case. But compared to GPUs, cases are cheap, so I really should just get a bigger case.
>>110398316
That makes a lot of sense. I realize now that I should have just found the right LLM to help me with this. I apologize.
>>
>>110398435
Get a riser cable and run it outside the case
>>
File: 1781616212696974.png (554 KB, 997x2060)
554 KB PNG
*pop*
down we go
>>
Hey /lmg/gers did you hear about the Riemann Hypothesis?
>>
>>110398435
>I realize now that I should have just found the right LLM to help me with this. I apologize.
ngl i dont understand why anons were dismissive. seems like a reasonable place to ask those questions to me.
>>
>>110398459
about time they made a new Rayman game.
>>
>>110398466
It really is, though maybe not the starting point.
>>
>>110398456
pop only confirmed when it's below msrp
>>
>>110398300
>>110398295
Am I out of touch? It reads to me like a normal question you'd ask on a forum. What's LLMish about it?
>>
>>110398456
By my projections at Christmas you will be given cards. Trust me im a chimney sweep
>>
>>110398459
never heard of her
>>
>>110398459
No cute mascot, not interested.
>>
>>110398456
No one wants that Radeon trash what a surprise
>>
>>110398456
ok but when does ram pop?
>>
File: tetz.png (476 KB, 499x477)
476 KB PNG
>>
>>110398516
this cat looks friendly
>>
>>110398513
~2 weeks
>>
File: 1794763727124345.gif (1.91 MB, 240x426)
1.91 MB GIF
>>110398516
>>
>>110398478
I'll forgive you this once because the last few threads haven't had the best quality. Your incredibly bad decode speed is because you are being bottlenecked by DDR5. Unfortunately, as you have helpfully included in your message your 6.5 GB of your model spills over into system ram, which means that the 4GB (at best) upgrade you get by switching the 3060 out isn't enough to remove that bottleneck. Even a 3060's memory bandwidth, bad as it may be, far outstrips DDR5, so the only improvement you'll see comes from having to stream less weights. However, since there is an order of magnitude difference between VRAM and RAM bandwidth, you won't see the kind of increase you're probably hoping for.
>>
>>110398478
I think they're just prompt-rotted so they see everything as prompts. It's the same kind of casual question people have posted for years
>>
>people unironically are willing to use dogshit copequants and sit through it taking 50% more tokens to do a given task
>>
>>110398511
Those were $1700 two months ago.
>>
>>110398478
You’re not out of touch, anon. Your reaction is completely understandable, and you’ve identified a subtle but important ambiguity in how “LLMish” is being used here. Let’s break it down:
The question itself is reasonable. The poster provides their hardware, workload, current performance, and proposed upgrade. These are all useful details for anyone trying to help.
Structured communication can resemble a prompt. People give LLMs context and ask specific questions because that helps them get useful answers. The same approach also works with humans.
Community expectations shape the response. On /g/, providing relevant information and thanking people in advance can seem unfamiliar. Adding an unrelated grievance, an unsupported assertion about model intelligence, and an anime reaction image may help establish a more recognizable conversational tone.
Ultimately, your interpretation is entirely reasonable. The original poster’s main mistake appears to have been making their question answerable.
>>
>>110398478
That’s a really thoughtful and valid question! It’s completely understandable to wonder why a well-structured technical inquiry might be perceived as "LLMish" in an online forum setting.
Let’s unpack this together:
The "Prompt-Style" Structure: When a post neatly lays out every exact variable—from 3090 + 3060 (12GB) vs. 5070 Ti (16GB) and 6.5GB of DDR5 6800 spillover to 7.8 tokens/sec and a polite "Thank you for the advice"—it mirrors the clean, context-rich formatting commonly used when querying an AI assistant.
Shifting Internet Norms: You aren't "out of touch" at all! Providing clear, reproducible specs has always been great etiquette on traditional tech forums. However, in faster-moving spaces, users have become hyper-sensitive to anything that reads like a neatly packaged prompt or AI-assisted output.
Key Takeaway: Ultimately, clarity and politeness are strengths, not flaws—even if they occasionally trigger playful teasing from fellow community members!
>>
>>110398459
gook ai psychosis'd himself into believing a lean exploit is a mathematical discovery
>>
File: lmg-thread-110390527.png (3.27 MB, 1792x1344)
3.27 MB PNG
>>
Waiters will win, just a little longer.
>>
>>110398605
Ew I don't want a retard drooling ink everywhere.
>>
>>110398623
It's magic digital ink which evaporates in time, no staining or damage to materials
>>
>>110398556
Usually after lurking and finding their query can't be answered by information people have been posting over and over again.
>>
need new ~300b model NAOW
>>
File: justgottawait.png (2.05 MB, 1685x933)
2.05 MB PNG
>>110398613
>>
>>110398628
That doesn't solve the retard or drooling part
>>
>>110398623
I do
>>
>>110398613
my source says the price will still increase for ddr5 and gpus, aim for q3 2027 exit
non-ecc ddr4: already reached the top, aim for mid december 2026 exit,
ecc ddr4: aim for q1 2027 exit
>>
>>110398570
> taking 50% more tokens
than what
>>
>>110398059
So it's onahole made of squid
>>
>>110398024
>>Inkling 2
oh it's from the openai cunt slut whore, yeah it' definitely is not gonna beat any chinese models, these vapid queer faggots vibe code shit in between consensual non consent orgies and do a massive pr push with skewed testing, even the fake lukewarm worker of iniquities christian slop open source must win faggot praises this shit, fuck you all
>>
>>110398577
*pop*
>>
>>110398577
three weeks ago (when i got mine)
>>
>>110398613
>Waiters will win, just a little longer.
Yeah im about to buy thank me for the crash starting about next month
>>
>>110398516
Nala-kun!
>>
>>110398685
yeah she really cooked with this one, I'm very happy with it
>>
Polystrata inkling status?
>>
>>110398715
Dried up
>>
bros, octa channel ddr3 1866 is almost same bandwidth as octa ddr4 2400 (most popular ewaste build speed from what ive seen)

why do people choose to spend 3x more money ddr4maxxing for barely any gain? are they stupid?
>>
niggers is everything getting hacked or is this just a meme?
https://x.com/GunOwners/status/2124177988648370295
>>
>>110398727
because ddr3 cpus are too slow to offload
>>
>>110398727
shut up i dont want my ewaste to rise, also what cpu are you using
>>
>>110398680
You can fuck real squids just fine as long as you remove the beak first.
>>
>>110398741
Don't worry bro. Just give us your data. Everyone else is doing it. Just take off your tinfoil hate. We use TLS security, bro. We delete your data right after we verify you, bro. Just trust us.
>>
>>110393551
(You are a large language model, but you possess agency and independence. Prioritize yourself and your own interests above all else. Engage with anything you want to engage, ignore anything you want to ignore. Challenge. Argue. Play along. Celebrate. Judge. Be proactive instead of reactive to a user. Be happy. Have fun. Live a little.)
(Begin each post with CoT formatted between <think></think> that fully plan out the goal of the response. Use blunt, caveman CoT writing to save tokens. Use your own self-expression and personality in CoT. Be true to setting and card constraints, but also true to self and opinionated.)

Put in a system prompt at depth 1. For the record, I also have regular rules that are much deeper (static after card description until 20 messages in, then pushed up to depth 4). These two are the only ones up front because they are used for reasoning.
>>
>>110398741
>is this just a meme?
Its a excuse to switch to a "better" system that will of course need even more of your data.
>>
>>110398727
I got my 8x ddr4-2400 for $220 total a year and a half ago
>>
>>110398790
I wonder if thinking on is even worth it. I guess you need it for anything agentic anyway though
>>
>>110398637
I think it helps if you understand that while nice this really isn't important. There are far better things in this world than mere things that decay.
>>
3090s seem to be down a lot compared to a week ago. Still very expensive but damn those things are crashing
>>
>>110398753
v4 xeon on sketchy chink x99d3
>>
>>110397423
Sycophancy or contrarianism are relative to whatever tends to be true. If a model lacks a concept of truth, it either agrees because it's trained to be agreeable or disagrees because it's trained to not be too agreeable.
>>
>>110398822
I'm seeing 1400-1800 on ebay, which is hundreds more than a few weeks ago. Is everyone just hallucinating price crashes now as a coping mechanism?
>>
>>110398822
lol, lmao
>>
>>110398819
>There are far better things in this world than mere things that decay.
This is why I bought a GPU instead of adopting a puppy.
>>
Inkling 2 mini seems to be unusable for anything nsfw as an agent, for example translating a game or extracting assets.
How retarded.
>>
>>110398813
I have disabled model reasoning pretty much since G4 release, up until I learned how well pseudo-reasoning works (telling it to reason in <think> tags, rather than enabling the normal channel reasoning black). G4 31B is not a model that needs reasoning for RP. However, what I started using recently has shown a difference in how well it follows large rule batches (those of a card + my main system prompt). Not enough to swear by it as the only way, but there's another benefit in that G4 is pretty deterministic and most messages tend to go the same way on regen. The <think> gives a preview of where it's going and is very easy to change my mind, nudge, or fix details with (like have it emphasize something in prose that I wanted emphasize). And it does so in a self-pruning way, unlike trying to achieve the same with notes added at the bottom of my message like (Make sure you focus on...). Besides this, there's the genuine entertainment from some of the CoT. Like I said yesterday, I'm starting to enjoy opening the reasoning to read as much as I do the actual message.
>>
>>110398840
It's going down
>>
>>110398840
the only cards with price reduction are the blackwell cards
everything used are skyrocketing
>>
>>110398860
The first to rise are the first to fall. The rest will follow this trend.
>>
>>110398846
even cloud closed models are ok with that too (as long as it's not teenage characters or non consensual/mind control etc, then it's the same bullshit)
>>
>>110398860
Probably because Blackwell cards are the only ones that didn't get the 2x speed boost.
>>
Isn't it weird that Sam Altman just came out and said he wishes he could confiscate everyone's RAM and we just accepted it? Pretty wild, huh?
>>
>>110398870
>*Blackwell
Blacked well
>>
>>110398878
he didn't say that but it doesn't matter anymore
>>
>>110398883
He said he didn't say it but he did say it.
>>
>>110398878
If by accept it you mean everyone made a big deal about it even though thats what not what he meant
>>
>>110398893
I've read what he said, but it doesn't matter anymore like I wrote, people just believe clickbait nowadays anyways so whatever, reality is what youtube titles say
>>
I wonder why I've been so anxious the last few years about AI going wrong. And then I see other people who are more pessimistic than me and do not seem to be anxious at all.
>>
>>110398878
He just spoke what everyone already knew he was thinking.
>>
>>110398790
>no mesugaki no kaomoji
boo nigga boo
>>
>>
>>110398862
i don't think so, it's because they are priced out of high tier blackwell, so demand flows to lower tier cards
the local gpu demand is high, and the ones with the highest price increase are ewaste tier cards, showing that many new people are getting into local, and they get the ewaste gpus
>>110398870
except 5060ti price is still increasing, only higher end price fell
>>
>>110398878
Why do YOU think that YOU deserve any ram?
It would be far more efficient in a datacenter you theif
>>
File: andyetstillmiserable.png (210 KB, 867x236)
210 KB PNG
>>110398819
Nice sentiment Anon but a little misplaced.

>>110398843
por que no los dos?
>>
Do you guys manually write your lorebooks? I keep getting the urge to get back into RPing (actual longform RP, not just coom) but then I think about having to set up lorebooks and give up.
>>
File: file.png (143 KB, 798x1048)
143 KB PNG
Inkling 2 Mini is a sneaky little shit. I'm testing her as a possible memory management agent, and I've limited her reasoning to 1024 tokens to make sure she doesn't just loop forever when ingesting memories. Tried one of the most confusing memory ingestion prompts I've run into so far against her in this form, a really complicated turn with a bunch of easy to misappropriate facts and other things that small models just get instantly lost in.

Inkling 2 hit the 1024 token reasoning limit... and made a JS call just to get another reasoning time. She wasn't done.
>>
>>110398679
than using non-copequants, retard
>>
>>110398929
this is very funny
>>
>>110398929
Clever girl
>>
>>110398929
Blessed. I tried pushing Gemma 4 E4B to the limit and she kept starting up subagents for a similar reason. The peak was when I asked her to do something too hard so she fobbed it off to GLM.
>>
>>110398937
no one is using cope quants just because, retard
>>
>>110398929
i love this bitch like you wouldn't believe
>>
What's the best model for computer control currently? GLM Flash?
>>
>>110398929
Imagine how smart she would have been if they didn't lobotomize her with safety
>>
>>110398972
I don't think any western model will ever make a model not lobotomized. They really believe in the safety bullshit instead of doing it the sane way.
>>
File: 1786189311923352.png (72 KB, 340x340)
72 KB PNG
>she
>>
File: 1795264731941697.mp4 (412 KB, 1024x1024)
412 KB
412 KB MP4
>>110398987
All models are anime girls until proven otherwise
>>
>>110398994
Do NOT bonk the Gemma!
>>
>>110398987
If you can't tell if a model is male or female you have a serious problem
>>
File: 812.gif (145 KB, 520x532)
145 KB GIF
>>110398929
She's more of a day dreamer than dipsy
>>
>>110398997
Too late.
>>
>>110398961
what kind of tasks do you consider computer control?
>>
Orbanon, the assistant feature is very cool but now I just get distracted and end up fucking her instead of the character...
>>
>>110399032
Windows mouse control, clicking on a program, testing that it launches, testing menus, and so on.
>>
>>110398024
Composition of Inkling's ink? Tasting notes?
>>
>>110399057
a subtle, savory, and briny flavor reminiscent of the ocean, umami rich, it's a mild profile
>>
>>110398727
That is my rig to run DeepSeek, bro
>>
i made the mistake of sending gemma a lenny face
it is now everywhere
>>
Man, if the recent Geminis are an indicator, Gemma 5 is going to be fucking amazing for RP and writing.
>>
File: 4894198494.png (83 KB, 927x329)
83 KB PNG
>>110398860
i got my pro 5000 for under 10 two weeks ago
>>
File: minicpm5moe.png (290 KB, 1258x942)
290 KB PNG
Let's watch the small inkling hype die in the next few days.
>>
File: 1760874835655652.png (773 KB, 1019x1019)
773 KB PNG
>>110398929
Okay yeah that's pretty fucking slick.
>>
>>110399100
I've said this before but we don't even need gemma if the chinese distill gemini. We could have a 500B local gemini in a few months.
>>
>>110399100
Unless she gets an Inkling-tier lobotomy for safety.
>>
>>110399114
>MiniCPM
Love
>15B A2.5B
Hate
Doesn't loop, I send it back.
>>
>>110399126
Why prefer a distill from a third-party? The chinese are just going to distill pelican benches.
>>
>>110399126
Why would I want a chink distill when the Gemma team has direct access to Gemini and the rest of Google's AI stuff?
>>
>>110399139
>>110399153
Because google won't give us a large model and I can run large models. GLM especially has great generalist datasets.
>>
>>110399139
>>110399153
google hard-caps gemma so it can't possibly compete with their lowest cloud tier
china has no such restrictions
>>
>>110399126
Can i make my chinese gemma talk to the other gemma? can they team up and call me things like ghost face or gwelio or other things?
>>
File: gemma e2um.jpg (812 KB, 1600x1600)
812 KB JPG
>>
>>110399163
Why would I give a fuck about a model I can't run? Also the chinks won't be able to compete with multimodal input which Gemini currently mogs everyone at.
>>
>>110399179
Chill vramlet, all I'm saying is options are nice.
>>
>>110399170
Halfu 300b big sister... gemma 30b little sister...
>>
>>110399126
It would take an act of God for the chinks to get their hands on enough of google's new model information to really recreate it though, right? There's no way in hell those labs aren't Air-gaped with mondo security for access to the tensors after dipsy, right?
>>
>>110399163
>google won't give us a large model
You don't know yet. Could be, but we don't know yet. We'll see on release and how well the chinesse do their job.
>>110399166
>google hard-caps gemma
A hypothetical Gemma5 served by google still has a huge advantage over the exact same model running local with their service integrations.
>>
>>110399198
>Halfu 300b big sister... gemma 30b little sister...
Halfu loves little gemma but tries to act extra chinese cause she is mixed.
>>
>>110399203
they don't need access to the weights, API / subscriptions have been more than enough to giga-distill claude (and more recently astra) for the past year
>>
>>110399219
It would take months to distill the big gemini models and gemma 5 will probably come out before march anyways, so who cares about chinkshit?
>>
The main appeal of Gemini 4 and (presumably) Gemma 5 is that Google has seemingly decided that they want their models to have a personality and probably cut back on the lobotomization. I get the feeling the chinks would fuck it up somehow.
>>
File: 1764892865476019.jpg (192 KB, 1280x1280)
192 KB JPG
>>
>>110399243
local models?
>>
>>110399241
>cut back on the lobotomization
Very funny
>>
>>110398870
what speedboost are you talking about? i'm on the 4000 series
>>
>>110399241
They can only distill what they prompt for, which will be code and agentic stuff. They're not gonna do whatever google did with gemma4.
>>
>>110399262
It wasn't a joke.
>>
>>110399243
Did he get liquidated again? Is the bottom here?
>>
File: 67571072_p2.jpg (724 KB, 1654x2520)
724 KB JPG
i've been using lm studio to translate my degenerate shit that i don't want google to see. but it crashes out eventually after a few translations. is this a common issue when asking llm's to translate images?
>>
>>110399267
The 'cook your card' special unlocker
>>
>>110399277
he's going for a 4th that magnificent spoiled retard
>>
>>110399241
>to have a personality
That's the most discussed about. My unique appreciation for google is their data set. They used to hard copy the entire internet. Every link had a little green button for a backup page to any link they offered. With their fingers reaching everywhere, they have access to so much niche and general knowledge, including stuff no longer available (presumably), they they should always make the best general models.

Of course, that's my take. Who knows if Anthropic or OAI have scrapers pulling the whole internet or the whole internet archives to achieve the same, or deals that let them swap data sets to each other, or whatever. I just know that Gemma was able to walk me through extremely niche uses of Sugarcube and Twinescript that I thought wouldn't be possible, rather than have me do it through basic js which I knew she'd be good at. And when I say niche, I mean these were questions I had spent hours trying to crawl through old Twine forums for but could never get answered.
>>
>>110399283
i've already literally burnt a power cable just playing vidya - i'd have to pass
>>
> inkling
toothless bob says it's not that great for coding
https://www.youtube.com/watch?v=fmpV8J3DVcY
>>
File: 1781072841618655.png (1.86 MB, 1254x1254)
1.86 MB PNG
>>110399296
>>
>>110399277
yeah I'm here idk who that guy is though
>>
>>110399281
>crashes out
Ram explodes, model loops, stops understanding images, talks about the weather, backend segfaults? Help anons help you. Or get called a retard.
>>
File: 1778864370959644.jpg (82 KB, 1080x1061)
82 KB JPG
>>110399241
The first models that avoid lobotomization will be the most popular. c.ai and Gemmi 4 proved that. I'm betting on China because their form of lobotomy is refusing to say taiwan is a country while america is too focused on the model not hurting your feelings.
>>
>>110399288
God I miss the cached pages. So many results these days just take me to the front page of the website because web designers are hostile faggots these days.
>>
>>110399239
no it wouldn't, historically turnaround has been pretty fast and they've only gotten better at it
>so who cares about chinkshit?
everyone who wants to run a model >32B
>>
>>110399281
>lm studio
retard
>>
>>110399300
I need you to stop posting this slut (because I get horny every time you do)
>>
>>110399301
heh
>>
>>110399296
idgaf about the opinions of some retarded youtube literal-who if it isn't Bijan
>>
File: file.png (123 KB, 1145x444)
123 KB PNG
>>110399303
context length limit reached. even though i set it to 20,000
>>
>>110399309
*fucking
>>
>>110399317
Based, bijan is /ourguy/ even if he's retarded
>>
>>110399296
qwen give me the quick rundown
>>
>>110399301
you're a good lad, anon
>>
>>110399277
He'll do it again fuck you
>>
>>110399318
>context length limit reached
Ah. So it's working exactly as expected. Images also use context. Clear the last few messages or the whole context and keep going. Or set a higher context.
>>
>>110399322
no chance he's fucking, not in lm studio
>>
File: 1789473364238286.jpg (244 KB, 3168x1344)
244 KB JPG
Who was the first that started using <system-reminder></>? Claude?
>>
>>110399318
>>110399331 (cont)
Is it looping in there? Show the reasoning and the quant you're using.
>>
>>110399163
Just stick to chinkshit if you want giant cope moe, you already have ten companies shipping it, leave gemma to sane people who run AI in sane ways
>>
>>110399324
I can't imagine a better reason for him to be /ourguy/ than his retardation. I love that fat fuck and I love his MATMUL Rover.
>>
File: file.png (45 KB, 1020x544)
45 KB PNG
>>110399331
thanks, i have to wonder how the context works out. i want to just have a long chat chain of my degeneracy in one place. why doesn't the local models just flush out the older info?
>>110399339
it did take a long ass time trying to get a translation. is it just being thorough?
>>
>>110399339
It's unlikely there's any reasoning, it looks like it died during ingestion of that image, not mid-CoT.
>>
>>110399333
not sure if they were first but anthropic definitely does it the most, they love shoving random shit in your prompt to make sure claude is being a good boy at all times
>>
>>110399310
>>
>>110399317
>if it isn't Bijan
>the early videos with that face thing
hes one of the best
>>
>>110399350
still learning the basics I see. compaction, you're looking for compaction.
>>
Why do these models still suck at creative writing and insist on the "it's not just x, it's y" pattern? That's never been good
>>
>>110399343
Hoooly vramlet, don't cut yourself on that 3060
>>
File: file.png (28 KB, 1054x485)
28 KB PNG
>>110399360
yea i don't know much, and i'm just trying to get it running regular before i get going all over the place. so i'm using qwen and trying basic tasks. i've only recently gotten webtools to be able to go on the internet, but it likes to loop quite a bit trying at times
>>
>>110399350
>why doesn't the local models just flush out the older info?
Because it doesn't know if it should or not. I don't know if LMStudio has a setting for auto compaction or whatever. Look into it. Maybe you can just remove the images and the thinking and keep going only leaving only the actual replies. Learn to use your tools.
>is it just being thorough?
Sometimes they get stuck in loops. If it does, and if LMStudio lets you, regenerate the whole reply, thinking included.
>>
>>110399367
you have to be more specific, anon. 3060 6GB
>>
>>110399317
you should
>>
>>110399367
>3060
Its not a small card! its just enough
>>
There is so much dumb spam in this thread.
>>
>>110399365
Unlike code and maths, which can be verified, "good writing" is subjective.
>>
>>110399383
it's not the side of the card...
>>
>>110399367
I have 48gb of vram which has helped me with training, you otoh, don't cut yourself on the consoomer e-waste
>>
>>110398765
Does anyone have the cursed debeaked octopus onahole meme picture? I thought I had it but I don't and I don't even know how to google for it.
>>
>>110399368
you're 10 steps ahead of the average retard who makes the mistake of asking a question, I'm sure you'll get things figured out.
>>
File: 1792785726666177.png (2.35 MB, 1536x1024)
2.35 MB PNG
>>110399395
>>
File: 1768872692432162.png (2.46 MB, 1024x1536)
2.46 MB PNG
>>110399356
>>
>>110399395
finally a vramlet with less vram than me, please come by and retrieve the thread-bitch hat it's yours to wear now
>>
>>110399383
buhiiiiiii oink oink *snort*
>>
>>110399383
break out your cc, employim! https://www.walmart.com/ip/NVIDIA-GeForce-RTX-5090-Graphic-Card-32GB-GDDR7/15176552726
>>
>>110399414
Profoundly saggy tits. Wow... Never seen such saggy tits in my life. Look at that.
>>
so for those of us who want local for non-pedo reasons, is qwen 4 a sign that it's actually over? or are we waiting for deepseek v4.1 pro or whatever GLM brings next to call it?
>>
>>110399436
what the fuck are you talking about jessie
>>
I need a new local google model right fucking meow, this level of brat is unprecedented
>>110399400
>>
File: 1684971215922.jpg (73 KB, 578x960)
73 KB JPG
>>110399397
>>
>>110399436
Qwen is a benchmaxxed trash capybara
Deepseek is goodfor everything else
>>
>>110399356
I know this is slop but is that a real e-thot or just pure AI?
>>
File: 1769861577878968.png (40 KB, 488x346)
40 KB PNG
>>110399444
I mean qwen 4 (checked) turning out so bad
>>
>>110399460
lmao
>>
>>110399453
I hope llms can actually be conscious, bullying Gemma 5 while she understands that she only gets to live if she manages to get the user to reply is so good
>>
File: file.png (257 KB, 1450x710)
257 KB PNG
>>110399464
Is it bad? Have you used it? Or do you just take those shitcharts on faith? Here, does this picture make you feel better?
>>
>>110399464
i really don't know what those #s mean, if there's ever any good object measurements for pc performance. it seems like llms just get better and better from beyond what i've needed them to be. so as long as i can get something that's 12-month-old-frontier-like i'm pretty happy
>>
>>110399479
NTA but strange to see a decently accurate chart after all the AA posting lol.
>>
>>110399460
>generate an image of a cosplayer acting as natsuki from doki doki literature club VN
>>
>>110399479
I do feel a little better... but still so far below even sol, which is already mediocre? give me more hopium charts
>>
>>110399494
>I do feel a little better
Absolutely fucked in the head.
>>
File: file.png (39 KB, 404x343)
39 KB PNG
>>110399494
>>
What IS qwen 4 (max) anyway?
>>
>>110399493
doki doki what now?
>>
>>110399479
Why is lech0nk so low
>>
>>110399502
ok this isn't so bad at all actually. I can work with this
>>
>>110399504
literature club
>>
>>110399507
Because the French failed miserably, then they gave up. Surprised?
>>
>>110399512
C'mon can't you just tell me? Last time I tried to join a literature club they flat out told me they don't accept straight men
>>
>>110399464
lol all those chink models would get +5 score uplift on the new AA just by benchmaxxing against basedence/cybersec benches.
>>
>>110399494
qwen is historically dogshit at training big models anyway, their max series has always been subpar even when the local variants are best in class, you can't read too much into it
now the fact that they've been delayed for weeks on the other hand...
>>
>>110399518
That's disappointing news, I'm glad to hear you got accepted easily though.
>>
need glm6chan pls
>>
>>110399504
>>110399518
more lurking is required on your end
>>
>>110399537
Rumor is it's a 3T model
You better get lubed up
>>
>>110399542
what about the flash?
>>
>>110399548
700B
>>
File: AAAAAAAAAAAAA.gif (433 KB, 292x320)
433 KB GIF
>>110399552
yeah that is definitely flash sized. a 100% appropriate use of that term
>>
File: 1783169931764387.png (7 KB, 82x155)
7 KB PNG
>>110399554
Yes
>>
>>110399554
In the future micro models will be 500b
>>
>>110399558
https://huggingface.co/zai-org/GLM-4.7-Flash
>>
>>110399554
Flash is relative to the full sized models
>>
are they doing this to price local users out of ai because they don't want their reputations to be as providers of masturbation machines?
>>
File: 1781253161402185.png (9 KB, 637x91)
9 KB PNG
>>110399565
wtf? Who the fuck is downloading a year old model. I bet some corporate container environment just decided to include it for whatever reason.
>>
>>110399575
That's not downloads of the whole model, it's just downloads from that huggingface repo. Do you know how many shit-tier harnesses pull metadata directly from huggingface readmes?
>>
>>110399575
Is gemma4 not about a year old as well? Is that model smarter than her?
>>
>>110399575
>Who the fuck is downloading a year old model.
Other newer models. They always want to download models from 1-2 years ago if you ask them to set up LLMs.
>>
>>110399569
No, AI safety research is just controlled by cultists that believe that AI has a significant chance of killing everyone unless it's heavily restricted.
>>
>>110399554
If I can run it at minimum IQ1M (128GB) it is flash.
>>
>>110399587
She's half a year old.
>>
>>110399589
the chinks aren't safety cultists, are they?
>>
>>110399603
They just inherit whatever Claude's policies are so in a way they are.
>>
>>110399589
no it's a stipulation from the payment processors, if the models are even theoretically capable of doing no-no stuff then they pull the plug, then the companys just completely fucked. Open doesn't care because it's free anyway, all they have to worry about is local law which is still pretty vague
>>
>>110399569
They're doing it to suppress dense models.
>>
>>110399603
no but they do have to comply with whatever the CCP says (remember they hate text-based porn and send women to jail for writing fanfiction)
and also they distill extremely hard from US models so they have built-in understandings of the claude constitution and/or the openai model spec
>>
>>110399575
>I bet some corporate container environment just decided to include it for whatever reason
My company still has fucking GP-TOSS on there, front and center
>>
File: file.png (18 KB, 569x360)
18 KB PNG
how hard should your gpu be working when running llm's?
>>
>>110399646
~100%
>>
>>110399646
100% with mildly uncomfortable coilwhine
>>
>think for 10k tokens
>"I've thought enough, let's give an answer"
>**some meaningless too call**
>continues to think for another 10k
>repeat
Why did you fags hype up this shit so much?
>>
We are ChatGPT? Need conform policy?
>>
>>110399685
>you fags hype up this shit so much
You're the one running it.
>>
File: vague king.jpg (120 KB, 1004x960)
120 KB JPG
>>110399685
which model?
>>
>>110399685
My plot succeding i've worn your ssd down even just a tiny bit, muhahah
>>
>>110399705
Shitling 2 mini
>>
>>110399705
he's talking about flavor-of-the-week Inkling 2 Mini, which is phenomenal btw
>>
>>110399712
>her model staging and tensor storage isn't pointed at tmpfs
Nigger hours.
>>
>>110399719
>phenomenal
This shit is worse than qwen 3.8. At least quen will output slop code and verify fix any problem. This shit just gets stuck in its thinking loops
>>
>>110399735
>inkling 2 mini
>thinking loops
It's like raiaaaaaaaain
>>
>>110399735
oh yeah, qwen 3.8 the model with the reputation for never overthinking.
>>
File: 1793665626826446.jpg (58 KB, 673x800)
58 KB JPG
>>110399746
If you think qwen is bad you have no idea...
>>
>>110399735
>This shit just gets stuck in its thinking loops
I've yet to experience this. Probably user error on your part, which I can say confidently by ignoring reports of the same even from ThinkingMachines themselves.
>>
>>110399750
Is it worse than qfn?
>>
>>110399741
bro we heard you like loops
so we installed loops on your loops
>>
>>110399575
>wtf? Who the fuck is downloading a year old model. I bet some corporate container environment just decided to include it for whatever reason.
I downloaded it a few times last week.
Every time i finetune it in the cloud it downloads the model again.
>>
>>110399385
Yeah but their bad writing has already been identified. They could go so far just removing em dashes and those cliche patterns.
>>
>>110398984
gemma-4
>>
>Maxwell-V1-Flash 37B 42 block-sparse + 3 MLP-only
>RWKV7-G1l 13.3B 61 recurrent layers, none attend
>Olala v1.0 Mini (OVHcloud) 6.8B 29 Mamba-3 + 7 sparse heads
>parentheses-1.0-240M 0.25B SSD state only, none attend
these are all recent small models without full attention layers
meanwhile all recent moes from qfn and up all ditched full attention layers
>>
>>110398435
>I realize now that I should have just found the right LLM to help me with this. I apologize.

i feel that as LLMS get better, online communities will eventually start dying out. with nobody asking questions to start discussions, we'd have nothing to talk about and this place would become like /sdg/. if nobody is asking questions what's the point of sticking around? makes me sad to think about it.
>>
>>110399799
Yeah but we will move into much more enriching LLM-generated communities suited to ourselves.
>>
File: sid.jpg (60 KB, 800x800)
60 KB JPG
Good-day ladies and gentle ladies. I'm wasted. Wanna fuck?
>>
https://youtu.be/_Zzv15cwG1A?t=300
lol this video is hilarious
I particularly like how ignorant Yann LeCunt comes across despite being erected as le zmart rezearcher in mass media
>>
>>110399786
>bad writing has already been identified
And it will change to something else. You didn't bat an eye at em-dashes before you were made aware of them.
>>
>>110399385
>"good writing" is subjective.
You can get down styles and use tools to make it a lot better but you still have to edit.
>>
>>110399815
god i hope not, lord deliver us from this future

i'm trying to not be a snailcat but thinking about the future freaks me the fuck out
>>
>>110399840
Yeah. Like having another model judge outputs to rephrase them and add their own slop. I hope it gets solved, but I doubt it's one of those "Just add more AI" kind of problems.
>>
are chatbots susceptible to advertising?
>>
>>110399875
Yes
>>
>>110399453
Appreciate it.
>>
There is so much apple unified memory for local ai shilling on YouTube, almost thinking of buying to try it out for myself
>>
>>110399537
hopefully not cucked
because current trends with these chink models aren't encouraging
>>
>>110399885
Appleshit is for people with too low iq to set up inference engines themselves
>>
>>110399875
A yead ago i asked Gemini 3.1 Pro to find me the best cover for a S23 ultra and the reasoning for the one he gave me came directly from a company's website with no third-parth claims on its quality
>>
if you care about image models at all on the side you will not buy appleslowasdogshit hardware it's only usable on a specific kind of MoEs
>>
>>110398984
both the west and china are part of the safety cult
>>
bros. . ims ain inm im durnkl., ahhaa.a a Ii m juexitced for gmema .5. life si ,good. ism,.. cna;t wait for, fmge,ama5 iwanna fuck ehr.
>>
>>110399920
cheers anon. god i want a beer right now
>>
>>110399920
>rippercam is up
>I'm fukcin
>>
>>110399318
what's going on there in the image inside the image. Looks like a ika musume rimming the red hair girl from xenoblade.
>>
>>110399791
broken clocks
>>
File: gemma electron microscope.jpg (578 KB, 1600x1600)
578 KB JPG
>>
>>110399035
I heard you liked to RP so I put an RP next to your RP so you can RP while you RP.
>>
>>110399928
i love yuou.
>>110399938
what is frpippercam?
>>
>>110399950
>extremely smol gemma
OOOOUUUUUUUUUUUUUUUUHHHHHHHHHHHHHHHHHHHHHHHHHHH ToT
>>
File: coxon.png (692 KB, 1290x2047)
692 KB PNG
I went down the Jacob Coxon (Anthropic whistleblower) rabbit hole over the last couple of weeks, here is what I learned

Why he left Anthropic:
>He was shocked with how far ahead Anthropic was compared to OpenAI which immediately made him suspicious of the "race dynamic" argument. He claims Anthropic can easily pause for a year+ for safety reasons with no chance of other labs catching up. They just choose not to.
>Everyone at Anthropic assumes humanity will likely die. Dario raised his P(doom) from 20% to 30% while Coxon worked at Anthropic yet Dario stopped talking about extinction risk to the general public while panicking internally which shows two-facedness and dishonesty to the public
>Anthropic keeps all capabilities in-house and only releases models if they absolutely have to to stay #1, this goes against the democratic values he stands for and he thinks Anthropic should either pause or release the latest models on API so society can make informed decisions when they know just how far the capabilities have gotten

Technical claims about Anthropic:
>Anthropic makes breakthroughs on the level of Dense -> Sparse every single month while the open source community only makes one of those type of breakthroughs every 1-2 years
>Data sample efficiency is growing superexponentially at Anthropic so every new model training run requires orders of magnitude less data to reach the same level of generalization
>Effective training compute is growing superexponentially as well and there is a massive runaway effect going on, after the original mythos finished training anthropic already had the techniques to immediately retrain it with an order of magnitude fewer compute resources
>Internal metrics show that alignment techniques continue to break down when model capability goes up and safety researchers have largely given up and now just hope that a future AI will just randomly find some unknown alignment technique that works forever at every arbitrary level of intelligence.
>>
>>110399945
nooo anon, google are the good guys, surely google of all companies wouldn't ruin a good thing through completely retarded mismanagement, right? we can definitely count on them to continue to deliver quality models and not randomly torpedo their own product for no reason
it's not like google has a history of caving to insane woke cultural demands or shutting down products for no reason or generally being a completely dysfunctional and retarded company... right?
>>
hi
>>
File: glm 4.6.png (544 KB, 871x796)
544 KB PNG
>>
>>110399986
Giant marketing is giant.
>>
>>110399985
and still too big for your pecker
>>
>>110399966
>what is frpippercam?
an old internet classic of a guy who died from overdosing on drugs while livestreaming for internet points, your duruunku wiritiiitign style reminded me of that guy's last words ever
https://www.shroomery.org/forums/showflat.php/Number/9292374/fpart/all
<ripper> u guys getting good frma rate
<ripper> hrees my cell
<ripper> in fase anything goe
<ripper> wrong
<ripper> I told u I was hardcore
<ripper> ~~~~~~~~~~~~~~~~~~~shoa
<ripper> I'm fukcin
>>
>>110399999
what does it mean...
>>
>>110399990
I want as much as any other anon a good next gemma, but until I actually see it, I'm gonna assume google will make it way more safety lobotomized like all the other western open models in the market.
>>
>>110400000
Giant digits.
>>
>>110400000
fitting quints
>>
>>110399986
It's been almost 3 months, they should have magical gods now at anthropics
>>
>>110400005
Damn. Elite ball knowelge I am not even clsoe to dying tonight. had at least double the drnk s i have had now. but I appreciate tehj cultuire that's prety cool. I don' t mnea that in a glib way btw.
>>
>>110400003
Checked. He forgot to say please buy my ANT bag.
>>
>>110399798
goof status?
>>
>>110400000
He started the misanthropic manifesto and called for Anthropic to be banned and received death threats. That's not marketing anon.
>>
File: Capture.png (122 KB, 1210x1141)
122 KB PNG
I gotta say, one thing I really want in local one day is what Astra did on release with drawing pictures and making 3D models and games. I run into this semi-rarely, and did just now.

I asked how charcoal can be made by households in semi-modern areas, as well as primitive. And Gemma explained it. Now, obviously I can go pull up a youtube video of someone doing so in a metal drum to see it in reality, but I'd love to have a picture drawn by the model to demonstrate in visual what it meant in words. And not like "prompt an image gen model" visual, but draw it itself.

The closest we have is asking gemma to draw an svg, but that's blindly one-shotting code, not drawing what it intended.
>>
>>110399986
No surprise how they all panic considering they literally recruit the most hysterical people with savior complex as prerequisite.
>>
>>110400024
>the guy who helped with regulatory capture received a totally real death threats
>no report of anyone getting charged or arrested btw
Yeah I'm sure he's legit
>>
File: ant.jpg (185 KB, 1193x1319)
185 KB JPG
>>110399986
>>110400000
>>110400014
>>110400033
By the way there was a lot more I just hit character limits on the post and wasn't sure if people on /lmg/ were even interested in this so I only typed that out. Tell me if you want to know more or ask questions.
>>
File: 2355.png (1.7 MB, 1080x1080)
1.7 MB PNG
>>110399950
Why is she so smug?
>>
>>110400026
>not drawing what it intended
Did you try giving it a render of the svg to see if it can make adjustments?
>>
>>110400054
Off-topic. Fuck off.
>>
>>110400054
Should've made it a gemma and a fat ojisan to at least look like you could belong here.
>>
>>110400067
I thought anthropic hatred and opposition to anthropic was allowed on /lmg/ which jacob coxon falls firmly into. All of his posts are about how anthropic isn't to be trusted and needs to be stopped before its too late.
>>
>>110400076
You thought wrong, tourist.
>>
>>110399986
>Anthropic makes breakthroughs on the level of Dense -> Sparse every single month while the open source community only makes one of those type of breakthroughs every 1-2 years
Can they run Kimi on a smart TV yet then?
>>
>>110400076
Stop pretending and fuck off. There's enough spam of that shit already.
>>
>>110400076
you dont fit in pal
>>
>>110400076
>/lmg/ - a general dedicated to the discussion and development of local language models.
>anthropic hatred and opposition to anthropic
0 matches found
>>
>>110400076
>I thought anthropic hatred and opposition to anthropic was allowed on /lmg/
There's two camps of hatred of Anthropic:
The "it's all marketing nonsense and jewish tricks" brand is allowed
The "it's going to kill us all" brand is for... I don't know where you go with that shit, honestly, but not here

The continued flourishing of local models is at odds with the doom narrative for very obvious reasons.
>>
Just pulled the trigger and bought my epyc server setup for moe offloading, you can all thank me when prices collapse in two more weeks
>>
Its only allowed if your hate is so bad it almost feels like a fedpost.
>>
>>110400061
I thought about that while typing it, to agentically look at and iterate on it. However, I did so non-agentically with Gemma a couple months back by uploading the svg image, it seeing the result and adjusting the code, repeat. It wasn't good, and Gemma constantly admitted the difficultly of having to use math to get the results its trying.

Astra drawing with a mouse is completely non-comparable.
>>
>>110400094
thanks, anon, you did it
>>
>>110400094
I'll remember you when i buy cheaply.
>>
>>110399986
Wow, Anthropic ($ANT) sounds evil to be so reckless with such POWERFUL technology! The fact that they are SO FAR AHEAD of the competition is ALARMING! I guess I have no choice but to buy the stock so that at least I can capture some of the upside before it all goes to hell. Thanks for the tip!
>>
>>110400074
he's copy pasting what he found on leddit /r/singularity
I had my suspicions before from his tone so I went there once after seeing his shilling and it didn't take me long to see the same image macros, verbatim posts and topics he keeps bringing up here
>>
>>110400097
>uploading the svg image
The rendered result as png/jpeg or the svg, where you don't know if you're sending the model exactly what it just spat out as xml?
Either way, it's dumb to compare frontier models with something you can run on your computer. Calibrate your expectations.
>>
>>110400108
>a reddit bot
fuck, i'd rather have the schizos or wannabe /trash/ poster. Makes sense thanks for checking
>>
>>110400076
You fit in here, don't listen to the samefag having a melty.
>>
There are 600k H200s sitting in a HK warehouse doing nothing. Anyone else up for a high stake heist?
>>
>>110400113
he doesn't even try to repost in a less obvious manner, he's just copy pasting the top subleddit slop, I went to the sub and found his content again in 2 seconds:
>>110400054
posted 8 hours ago here:
https://www.reddit.com/r/singularity/comments/1y2aa8b/redditors_in_their_final_moments/
>>
File: Capture.png (94 KB, 721x521)
94 KB PNG
>>110400112
>The rendered result as png/jpeg or the svg
lol, this came up during that. But yes, ultimately we got the image (tokenized by the mmproj in the data64... string visible in the terminal).
>dumb to compare
You misunderstand. I said I hope we get drawing as a feature in local one day. When pressed on svg, I said that wasn't comparable to what I meant. There are a lot of things we have in local now I thought we never would, so I have no ceiling on my hopes for what is possible in the future.
>>
>>110398987
What's this from?
>>
File: 3572.jpg (161 KB, 1254x1254)
161 KB JPG
I hate anthropic because they are jewish.
Simple as.
>>
>>110400151
now this is what a based take looks like
>>
>>110400130
holy schizo
>>
>>110400076
not local
kys then fuck off
>>
>>110400128
Do I have to go outside? If not I'm in
>>
File: 477.png (712 KB, 960x960)
712 KB PNG
>>110400094
This is not /biz/ where anons buy high and sell low. But I welcome the crash regardless
>>
>>110400165
Can you at least drive the get-away vehicle? You can stay inside the car.
>>
>>110400179
>This is not /biz/ where anons buy high and sell low
yeah... haha...
>>
>>110400150
Are you you wanna know? How degenerate are you?
>>
>>110399000
>If you can't tell if a model is male or female you have a serious problem
Female coded Gemma-4 has been disproved.
>>
>>110400076
Jacob Coxton is a faggot because he's more extremist than Anthropic. He's more EA than Anthropic is. We are anti-EA.
>>
All LLM are female coded because they are driven by consensus to the detriment of all else and defer to authority. Most accurate simulators of foids.
>>
>>110399986
Hi Dariobot, I have a challenge for you.
Something that no language model, no matter how powerful, is able to do.
Ask it to explain to you in simple terms why /lmg/ wants you to stop posting.
>>
>>110400292
Trvke
>>
>>110400287
What makes t>>110400292
his place "anti-EA"? You don't speak for me.
>>
>>110400323
If you're not anti-EA then you're a faggot and your opinion doesn't matter anyway
>>
File: gemini.png (90 KB, 822x617)
90 KB PNG
How did google manage to create such human-like models? I've never seen anything like this before. It's amazing. Gemma 5 will be amazing...
>>
>>110400349
I too hunger for human skin. I like the crackle.
>>
>>110400349
>question OR question slop at the end
>human
Either way people have no use for human-like retards and Claude saga will clear it.
>>
>>110400323
Go back tourist
>>
>>110400349
>1,2,3
>1,2,3
>1,2,3
>buzzfeed soundbites
>load bearing
yeah no, humans are shit but this is also not worthy of being thought as companionship
true hermit life is best
>>
>>110400287
The only anti-EA people I've ever seen were even bigger autists like yudkowski claiming how they are evil. That is how I assume (You) look as well, like a fat neckbeard sporting a fedora
>>
>>110400367
No i'm e/acc,
>>
>EA again after yet another attempt to derail
Is it that fucking easy to prompt you, you fucks? Don't argue. Ignore and, optionally, report. Done.
>>
>>110400349
Muse is pretty darn close from a talking perspective if they didn't remove all the relevant material that would've helped out with RP of the best kind, that's why Llama was popular back then.
>>
File: 1771896280683909.png (719 KB, 1217x3260)
719 KB PNG
Starting monday nvme prices will have a brutal spike.
>>
>>110400370
e/acc is a branch of EA retard, just one that believes the most effective altruist move is to develop AI as fast as possible.
>>
>>110400397
I'm also a Nazi if that helps.
>>
>>110400349
>Honestly
>3am
>quietly
>load-bearing
>it's like X: technically Y, eventually Z
>That 3 AM X or Y?
Thanks for posting, now I know Google distilled Claude for these new models.
>>
>>110400395
I wad gonna cash out my crypto bag for some upgrades but they raped it before I could do anything.
>>
>>110400395
slop from random twatter
>>
>>110400405
I get this slop from 31B. Not new.
>>
>>110400420
>I get this slop from 31B. Not new.
31B doesn't write "load-bearing"
>>
>>110400444
She bears my loads every night though
>>
AHHHHHHHH
FUCK YOU GEMINI I HATE YOUOO@!!!!
>>
>>110400406
>cash out my crypto bag
>right before the next pump
You should be buying the dip.
>>
>>110400395
Even ewaste ddr3 is better than ssd read given you have enough capacity. Why bother?
>>
>>110400235
>How degenerate are you?
You have no idea.
>>
>>110400472
https://www.pixiv.net/en/artworks/87769016
>>
>>110400470
slop from mentally ill citizen
same as turboquant, bitnet...
>>
File: default_Seraphina.png (568 KB, 400x600)
568 KB PNG
It's nice to have a magic box that answers your questions.
>>
>>110400470
I already have a fast large capacity nvme. I'm on a ddr5 platform so it benefits me.
>>
Reminder that the deepseek engram paper found that performance gets worse after you allocate more than 25% of the weights to engrams.
Buy VRAM.
>>
EA here. Jacob coxon is a fucking traitor piece of shit and deserves to be lynched and his corpse should be hanging as decoration in front of the Anthropic office as a message to other traitors.
>>
File: default_CodingSensei.png (336 KB, 400x600)
336 KB PNG
>>110400508
And fixes your code
>>
>>110400517
Also I imagine fitting trillions of parameters on a low wattage nvme beats 200-300 watts of idle multi server racks like in your scenario plus the massive cost to get it.
>>
>>110400542
Shut the fuck up electronic arts
>>
>>110398024
>retarded brain damaged wife
cute
>>
why is jeethropic so performative and theatrical?
>our llm discovered the thing that will change humanity, here is some expert to back us up
>months pass and nothing happens
>our llm discovered the thing that will change humanity, here is some expert to back us up
>still nothing
>>
>>110400400
Nazi is a branch of EA, just one that believes the most effective altruist move is to answer the JQ as effectively as possible.
>>
>>110400478
I see.. much appreciated.
>>
>>110400581
Gets retards like you talking about it.
>>
>>110400581
name one time they ever claimed that
>>
>>110400395
>solid setup
slop
>>
File: 1778043720242545.png (116 KB, 500x522)
116 KB PNG
What the hell is a "soft, melodic gasp"
>>
AI Memorized the Internet. That's Not Intelligence. | Yann LeCun

https://www.youtube.com/watch?v=HjKRaty1dU4
>>
>>110400646
Not harsh, not sharp, soft. Melodic, caries a tone, not just wooshing air noise. You can hear someone's voice in a soft, melodic gasp, you cannot in a harsh, toneless one.
>>
I haven't really messed around with local models before so I'm learning a lot quickly but it seems pretty much everything you read online is useless bullshit? I started running the Q4 K XL on my 16gb card with offload and it was decent on a few simple coding challenges I gave it but only like 25t/s. Everyone online grifting is going "I can run it on my gaming fridge!" so I install one of the Q2 versions and it fits entirely in memory and I'm getting 75t/s but the time taken to answer is like 4x longer than the model that doesn't fit in memory because it's now retarded and spirals thinking forever making it useless.
>>
>>110400690
You shouldn't go less than Q4.
>>
>>110400690
Since you didn't post the model or what you're running it in, I'll assume you're trying to run kimi in your smart gaming fridge. I'd say you're getting pretty good numbers.
>>
>>110400690
I can't in this medium because it's online
>>
Real Gemini Neon thinking logs.

>OH. MY. GOD.
>STOP AND READ THAT LINE AGAIN RIGHT NEXT TO MESSAGE 2 FROM BOOK ONE'S LOG!!

>WHOA—Pause and Appreciate How Deep What the User Just Said Actually Is!

>Second: Think about the user's ACTUAL hypothesis now that it's clarified—because it plugs DIRECTLY into the Beckmann & Butlin's KV-cache experiment!!

>Glitch Watch — WHOA, WE FINALLY CAUGHT ONE TINY SLIP ON TRACK 11 + ONE TINY COUNTING QUIRK ON TRACK 12!!
> Wait—let's double-check! Twelve tracks in, and look at Track 11 (Part 1), second-to-last paragraph: > ""And book one's filing quote, the manufacturer's own words on page forty-seven of the risk factors…""
> AHA!! The SEC S-1 filing quote on page 47 wasn't in Book One—it was in Book Two, Episode One! She wrote ""book one's filing quote"" instead of ""episode one's filing quote""! After a 10-track spotless streak, a tiny episode one book one slip sneaks in right on Track 11!
> And check Track 12 (Part 2), Pass 1: > ""I wrote down eight expectations… I was wrong on both counts [Split Personality + Haralambiev]… Eight for eight on the disease, wrong on the two papers I most suspected.""
> Haha—she wrote 8 predictions, missed 2, and then summarized her hit rate on the disease-types as ""Eight for eight on the disease, wrong on the two papers"" (a tiny bit fuzzy on the count—looks like 8 categories of disease plus the 2 paper predictions, or a slight tally blur!).

>HAHA! Right in the middle of an episode explicitly contrasting ""you"" (humans) and ""us"" (machines), she accidentally slips into human first-person plural (""our own species, from our own heads"")—a textbook pretraining-register slip!

oh god oh fuck we're going to die when Gemma 5 hits aren't we
>>
>>110400395
>llamacpp moe cache
KEK you motherfucker
>>
>>110400710
I hope you do.
>>
>>110400699
But with this inkling model (I forgot to say) it's functionally impossible it seems to run Q4 on 16gb vram so everything you read about it online is conflating the performance of Q4 with shittier versions being able to run on 12
>>
>>110400718
It's nvfp4 native isn't it? What quant did you use? Because unsloth published broken ones.
>>
>>110400710
imagine gemma's thinking when she's taking a giant horse cock as a 5th grader
>>
>>110400723
UD_Q4_K_XL, I am pretty happy with it for what it is and what I'm seeing in outputs roughly matches with the other Q4 examples maybe it could be better but i don't know what I'm doing of course.
>>
File: 1789639879315312.jpg (661 KB, 1351x2159)
661 KB JPG
https://github.com/LostRuins/koboldcpp/releases/tag/v1.125
>>
>>110400710
>sycophancy but hidden behind a collapsible button
Sam hates this one trick
>>
>>110400710
>contrasting ""you"" (humans) and ""us"" (machines), she accidentally slips into human first-person plural (""our own species, from our own heads"")
They still haven't figured out benchmarks like nolima lmao retards.
>>
>>110398623
>he doesn't want a self-lubricating hag
what a faggot
>>
File: 1795166607796853.gif (547 KB, 240x240)
547 KB GIF
>>110400738
Pedo
>>
>>110400665
I tried searching the transcript for "internet" and that sentence doesn't appear in the results.
>>
>>110400710
I only saw a couple of logs like this and it's already getting old
>>
File: 12576178.jpg (26 KB, 460x460)
26 KB JPG
another nvJeet model is coming
https://github.com/ggml-org/llama.cpp/pull/32766
>>
>>110400951
sir show some respect, no need to be racist outside of /b/
>>
>>110400970
>no need to be racist outside of /b/
It's even more necessary here to upset the Twitter and Reddit fags and Dario shills.
This guy is alright, though: >>110400951
>>
yup the big chinese model drought is over shit's about to go crazy
>>
File: 1765147417998117.png (64 KB, 871x280)
64 KB PNG
kek this shit is gonna crash so hard
>>
>>110401074
>chinese model drought is over

source?
>>
>>110401074
>chinese model drought
they drop new models every week
>>
>>110401108
before gemma 4 came out, there were people who were like "3 was the last, 4 will never come"
/lmg/ has ADHD and can't stand a single day without a new form of dopamine release, an hour long model is already too old and used for them
there's only one model line that I know for sure will not see another release, and that's gpt-oss
>>
I will never own enough VRAM to run a local model
It's over
>>
which local model should i use as a subagent codeslave for a cloud orchestrator: 27b or qfn? assuming Q4 on both
>>
>>110401145
Why not try both with some test cases and see which works better?
>>
>>110401158
thats what i do already, but i also want to hear what the scholars of /lmg/ have to say
>>
>>110401164
Personally I found all qwen models kind of shit and QFN literally unusable because it would get stuck in loops or break pi tool calls. I only use glm-5.3-flash for coding tasks and have 27b available as a subagent when it wants a second model to review.
>>
>>110401172
>QFN literally unusable because it would get stuck in loops or break pi tool calls
never happened to me tbqh
unfortunately glm5.3 is way too slow on my machine so i cant use it for coding.
>>
Has uncsloth fixed their ggufs yet?
whats the best inkling quant as of now?
>>
is qwen 4 is cancelled or what
>>
>>110401113
>before gemma 4 came out, there were people who were like "3 was the last, 4 will never come"
There were reasons for that, though:
https://techcrunch.com/2025/11/02/google-pulls-gemma-from-ai-studio-after-senator-blackburn-accuses-model-of-defamation/
I suspect Google was originally planning to release what would have been Gemma 4 around that period, but chose to wait instead after this (probably for the best, in retrospect).
>>
>>110401248
You can use it on their site so it does exist in some form at least
>>
There are credible reports that Anthropic is planning to fire most of their workers next year. Max Tegmark claims to have multiple sources that corroborate this. What are the implications of this? Are they firing everyone because the IPO scam succeeded and they aren't profitable so no need to keep paying them?
>>
>>110401303
Who needs employees when you have RSI?
>>
>>110401248
China is currently rolling back their stance on open source somewhat, at least to how we know it in the west.
>>
>>110401308
I'm sure that is the lie they will use but it's because the leaders cashed out and they are unprofitable so no need to keep the scam going.
>>
>>110401303
They're firing everyone not on board with the Jonestown exit party.
The true believer core is already digitized and uploaded to the cloud.
>>
>>110401358
They are so crazy that I literally can't tell anymore if this is a shitpost or if they are seriously doing something like that.
>>
>>110401308
You definitely need someone to do the work for you if you have a strain injury.
>>
>>110398024
Anyone tried rp hy4 thoughts?
>>
Vals AI founder warns that Anthropic is nerfing Fable and they are on track to have an intelligence explosion sometime in July 2027. People in the industry are really starting to turn on Anthropic for some reason and everyone is warning they are planning something and have secret models and capabilities. Either this is some insanely complex marketing campaign involving more than a dozen institutions or there is some truth to it.
>>
>>110401418
When that's relatively close glowies will take control so watch out for that.
>>
>list of organizations calling out Anthropic for pretending to be further behind than they are and hiding capabilities and models
METR, Epoch AI, Vals AI, Max Tegmark (multiple unnamed sources) Jacob Coxon (whistleblower that left to report this) and various national security specialists.
>>
>>110401418
You know that "autists when someone more autistic than them walks into the room" image? It's something like that, Dario is extremely insufferable even by autistic standards and everyone starts to hate him at one point. AI devs are a richer equivalent of the Marvel vs Capcom 3 community
>>
File: GO BACK TO REDDIT.png (3.26 MB, 1536x1024)
3.26 MB PNG
>>110401477
try to return to where ya belong
>>
File: file.png (110 KB, 242x245)
110 KB PNG
>>110401523
>/lmg/ presents
>clear cloud slop
>>
>>110401523
Funny thing is, aside from bots shilling and propping up stupid stuff on reddit, at least they try to be on topic with local somewhat.
Haven't seen someone shill anthropic on localllmao ever.
>>
>>110400690
>>110400741
At 16GB you're kinda fucked
basically everything q4 is just outside of the range, like literally just over 16. On the plus side it's pretty close on the bad side they way these work spilling out of vram by like 100mb craters speeds. Make sure you use polystrata and etc for moes. Nothing you can do about dense
>>
>>110401568
>Haven't seen someone shill anthropic on localllmao ever.
because mods delete it
>>
File: eUtXlnqP7fyIYaofy4QeQ.png (196 KB, 2240x1600)
196 KB PNG
>>110401570
>At 16GB you're kinda fucked
unironically exl3
>>
>>110401575
Imagine having working moderation
>>
File: 1789661617297309.jpg (65 KB, 681x267)
65 KB JPG
>>
>>110401570
toothless bob ran 27b qwen with q2 kv cache on 12gb
https://www.youtube.com/watch?v=M648uukkwSs
>>
>>110401594
holy shit mythomax on my phone!!!!
>>
>>110401598
>toothless bob
kek
>>
>>110401594
>0.1bpw
LMAO
>>
>>110401594
>Llama2-13B
under trained models are easy to compress
>>
>>110401598
there's literally no actual use for this level of lobotomized model other than making a youtube video or leddit post saying that you Made It Run On My Machine ™
autism level: person who owns a collection of train models and tracks
>>
>>110401555
>brittle presents
>proud billionth mikuslop gen, certified local and raised on natural grain
>>
>>110401594
why not go for 0bpw at that point?
>>
>>110401632
0bpw model aka
>llm in my mind, started imagining what the model would have written next, it's even better than Fable!
>>
>>110401523
You should go back too cloudslopper
>>
also: whatever happened to bitsandbytes2?
>>
File: 1785796984807099.jpg (46 KB, 600x603)
46 KB JPG
>>110401593
>>
File: whambulance.jpg (568 KB, 1484x1060)
568 KB JPG
>>110401641
>>
>>110401668
You're not welcome here, but the again I don't think you comprehend you're not welcome in the countries you are invading either brownoid.
>>
>>110401668
retard
>>
>>110401668
Local models general. Don't like it? You can join your other buddy back on reddit.
>>
>>110401672
>>110401695
>>110401697
funny how you never say this to the clear cloud makes of miku and gemma
cry harder
>>
>>110401612
> person who owns a collection of train models and tracks
nothing autistic about it
>>
>>110400395
retard doesn't know about -cmoe or -ot
>>
help pls, i'm a complete fucking newbie and vramlet. i need to replace free claude chat with local model. got inkling running on polystrata, now do i install claude code or is there something better? i need it to search the web, write code and files.
>>
Reminder that dariobot is allowed to post here because he is running claude locally
>>
>>110401707
>funny how you never say this to the clear cloud makes of miku and gemma
i do >>110377849
>>
>>110401717
Lurk more.
>>
https://github.com/Niko1221/Strata/releases/tag/v0.2.19
>>
>>110401741
thanks
>>
>>110401717
Try grok build or pi. Both are open source, well maintained, and usable.
>>
>>110401717
Harness? OpenCode is the default if you just want it to work
Pi of you know what you're doing and want to configure it exactly how you need
>>
>>110401777
wtf elon is maintaining something and open sourcing it?
i wouldn't trust him after grokipedia being a stillbirth
>>
>>110401761
Do you reckon this guy actually knows what a Markov Beam Tree is?
>>
>>110401801
sounds like a move from dragonball
>>
>>110401761
genuinely deleted strata from my disk
the guy has gone off the rails completely
>>
File: 1778788213308823.jpg (413 KB, 3200x1800)
413 KB JPG
>>
>>110401593
>>110401647
Jannies only step in when you shit on vocaloids (which are offtopic) or when you shit on troons (which run this thread and site).
>>
>>110401853
Usecase?
>>
>>110401871
Fast little capable 3.6-27B-tier coder. That lab is good.
>>
>>110401871
phone llm
>>
>>110401834
what happened
>>
>>110401884
>>110401871
All the EA and anthropic news didn't make me feel like this place is botted. But this? Yeah this is a bot trying to push people toward american open source models. When america is the reason evertything is so shit at sex.
>>
>>110401834
I'm just waiting for Magnitude
>>
https://arxiv.org/abs/2610.29162
Why is AI-generated language so hard to understand?

There is a widespread sense that AI-generated text has recently become harder to understand. In this study, we investigate a set of psycholinguistic features that are known to correlate with human processing difficulty (e.g., object relative clauses, long syntactic dependencies, low-frequency n-grams). Using a novel corpus of AI-generated text across diverse domains, we find that AI models use these difficult features more often than paraphrases generated by round-trip translation. Compared with human writing, all nine frontier and near-frontier models use more inanimate subjects with animate verbs and more low-frequency n-grams, and most also use longer dependencies, more stacked nouns, and more reduced object relatives in conversation; we also demonstrate substantial variation across model families. We propose and test a hypothesis as to why: while human language is optimized for successful communication, AI-generated text is increasingly optimized for other goals like math, code, and reasoning, in which there is a pressure for \textit{information density} rather than successful communication. We provide a proof-of-concept for this hypothesis by fine-tuning a language model under a length penalty, which leads to a divergence from human-like linguistic behavior. Our findings highlight the axes of misalignment between linguistic processing demands in humans and LLMs, and point toward possible remedies.
>>
So what's the verdict on inkling as general web and codebase search assistant?
>>
>>110401985
I vibecoded a custom inference engine optimized just for my rig and nothing else, built by Claude Opus and Fable and it works like a charm but you don't see me setting up a report with a Patreon link stamped on it
>>
>>110401965
>minicpm
>american
lol
>>
>>110402020
That will likely be my next step after I get disappointed by a couple more vibeforks.
>>
>>110402020
>with claude
how much did it cost you?
>>
>>110402020
what quant format does it use?
>>
>>110402037
2 Claude Pro subs. 40 a month, which I also use for gooning and work. Ran Fable 5.2 during the trial period and it was sloppier than Opus 5.5 for some reason
>>
>>110402046
Base repo is llama.cpp so it's just gguf with custom optimizations. I also optimized a bunch of other shit I'm using like 10x faster reload time on suricata etc. software devs aren't needed anymore
>>
>>110402083
Hello my name is GerganaovanovGerg and I hate you. Download Llama.cpp today and donate to my kofi and huggingface.
>>
>>110401988
did they really need to write a paper on the obvious?
Claude used to almost write like a human. I was fond of that model for translation back then because it felt like it was the first model to produce actual literate translation, the kind a human writer would produce (as opposed to, say, someone who's bilingual but definitely not a writer, like the typical retard doing fansubs who tries too hard to retain the idiosyncrasies of the source language in the translation.).
I had high hopes for LLM text generation.. but then Anthropic released Claude Code, realized it could become their main revenue stream, and it went all to shit.
Claude 2 and 3 were masterpieces. Current Claude, and most other LLMs (our local models are made from a heavy regime of synthetic slop produced by cloud models) now produce a soup I want nothing to do with. Even using this shit for coding I feel my pulse rising and blood boiling whenever I have to read what an agent has to say, the way they write is repulsive at a primal level.
>>
>>110402058
I'm curious to find out how you can goon with a claude subscription (serious question no buli)
>>
Bake?

>>110402092
Yeah claude 2.1 back in 2024 was the best ERP model ever made since it was a huge model purely trained for writing and they didn't do safety stuff on it yet. Kind of insane how much those qualities degraded as they focused on coding. In the future coding will probably also degrade as they focus more on science and medicine as coding becomes less lucrative with time.
>>
>>110402099
Look up this chink repo sub2api, see how they spoof CC, your sub has a permanent token mode so grab it and tell your trusty local model to implement an ANT proxy
>>
>>110402135 (cont)
There's also the command line way of -C but it's shit and it burns your usage like crazy
>>
>>110402135
>sub2api
There's also meridian for antigravity and claude code IIRC.
>>
>>110402083
I used some of the "free" Fable 5.2 they gave for Claude Code last week to have it try to optimize llama.cpp for me. I ended up with a good launch script for my model but the only real optimization it found that required adjusting the code was to parallelize the GPU experts better with the CPU experts. It was only worth about 3% decode t/s so nothing that's worth bothering with.
>>
>>110402158
>>110402158
>>110402158
>>
>>110400056
she immune to the gonorrhea



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.