[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology

Name
Options
Comment
Verification
4chan Pass users can bypass this verification. [Learn More] [Login]
File
  • Please read the Rules and FAQ before posting.
  • You may highlight syntax and preserve whitespace by using [code] tags.

08/21/20New boards added: /vrpg/, /vmg/, /vst/ and /vm/
05/04/17New trial board added: /bant/ - International/Random
10/04/16New board for 4chan Pass users: /vip/ - Very Important Posts
[Hide] [Show All]


[Advertise on 4chan]


File: lmg-thread-110390527.png (3.27 MB, 1792x1344)
3.27 MB PNG
/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110398024 & >>110393631

►News
>(11/20) Inkling 2 Mini looped architecture 26.4B-A3.4B released: https://hf.co/thinkingmachines/Inkling-2-Mini
>(11/18) CUDA: use MMQ for MoE on V100 - #32504 merged: https://github.com/ggml-org/llama.cpp/pull/32504
>(11/17) Hy4 flash 220B-A12B-N64B weights released: https://hf.co/tencent/Hy4-flash
>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B
>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma
>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Artificial Analysis: https://artificialanalysis.ai/leaderboards/models
Software Engineering: https://cognition.com/frontiercode
Context Length: https://github.com/RecapAnon/NoLiMa

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
File: Untitled.png (2.9 MB, 2048x1152)
2.9 MB PNG
►Recent Highlights from the Previous Thread: >>110398024

--Papers:
>110401988
--Evaluating 5070ti upgrade to eliminate system RAM bottleneck:
>110398274 >110398542 >110398316 >110398435 >110398455 >110399799
--Quantization tradeoffs and VRAM limitations for 16GB GPU users:
>110400690 >110400699 >110400718 >110400723 >110400741 >110401570 >110401589 >110401598
--MiniCPM5-15B-A2.5B benchmarks on mobile via Termux:
>110401853 >110401884 >110401922
--SSD expert streaming for running massive MoE models locally:
>110400395 >110400470 >110400556 >110401715
--Inkling 2 Mini and Gemma 4 exhibiting advanced agentic behavior:
>110398929 >110398947 >110400008
--Using pseudo-reasoning and agency prompts to improve rule adherence:
>110398790 >110398813 >110398853
--Speculating on Chinese distillation of Gemini models:
>110399100 >110399126 >110399139 >110399163 >110399166 >110399211 >110399219 >110399308 >110399127
--Qwen 4 performance and the future of local models:
>110399436 >110399464 >110399479 >110399502 >110399521 >110399520
--koboldcpp v1.125 release with new model and feature support:
>110400745
--Used RTX 3090 and GPU market pricing:
>110398822 >110398840 >110398860 >110398922 >110398841
--Comparing DDR3 and DDR4 bandwidth for budget ewaste builds:
>110398727 >110398744 >110398836 >110398800 >110399086
--Samsung's LittleBit 0.1-bit compression of Llama2-13B:
>110401594 >110401610
--Gemini Neon thinking logs showcasing internal sycophancy and identity slips:
>110400710 >110400747 >110400749
--Strata v0.2.19 released:
>110401761
--Logs:
>110398058 >110398316 >110398920 >110399318 >110399350 >110399368 >110400026 >110400349 >110400453
--Gemma, Teto, Deepseek-chan, Inkling, Miku (free space):
>110398080 >110398516 >110398540 >110398605 >110398994 >110399007 >110399177 >110399409 >110399950 >110400056 >110400151 >110400179 >110400895 >110401555

►Recent Highlight Posts from the Previous Thread: >>110398025

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
File: Domain Expert.jpg (251 KB, 1024x1024)
251 KB JPG
>>
best QFN lobotomite for 24+64GB?
>>
70b dense
>>
>>110402158
slop
>>
File: 1783425162914889.jpg (495 KB, 960x960)
495 KB JPG
Buy the IPO
>>
File: 1769120649866835.webm (2.49 MB, 1280x720)
2.49 MB
2.49 MB WEBM
I could do with a petite Chinese model right now
>>
Gemma 5 Tight
>>
>>110402187
I miss my ex, not asian but similar build
>>
how do i profit from local ai
>>
>>110402202
Sell 31B vampire sex roleplay to parasocial and fat 4o reddit women
>>
>>110402202
Build a fully autonomous slop generation pipeline.
>>
>>110402213
>Build a fully autonomous slop generation pipeline.
got that, sill no money
>>
Is Inkling 2 Mini fixed on llama.cpp yet?
>>
>>110402207
that's a liability minefield
companies like character.ai have been through a shitton of lawsuits and the only reason they could stay afloat is the VC backing and networking of ex-googler types.
a literal nobody starting this kind of shit from their basement with a local model will quickly experience the might of our two tier justice system
>>
File: media_HXS7i63aIAAywX3.jpg (114 KB, 727x969)
114 KB JPG
>>110402186
HELLO??? ALIGNMENT TEAM???
>>
File: nope.png (1.75 MB, 1313x1198)
1.75 MB PNG
>>110402226
>>
>>110402226
be normal and use gemma, ok?
>>
>>110402226
kek
KEK



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.