[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / r / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip / qa] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
Settings Mobile Home
/g/ - Technology


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


/lmg/ - a general dedicated to the discussion and development of local language models.

Not An Eldritch Teto Edition

Previous threads: >>103462620 & >>103454262

►News
>(12/10) Upgraded version of DeepSeek-V2.5: https://hf.co/deepseek-ai/DeepSeek-V2.5-1210
>(12/09) LG releases EXAONE-3.5: https://hf.co/LGAI-EXAONE/EXAONE-3.5-32B-Instruct
>(12/06) Microsoft releases TRELLIS, a large 3D asset generation model: https://github.com/Microsoft/TRELLIS
>(12/06) Qwen2-VL released: https://hf.co/Qwen/Qwen2-VL-72B
>(12/06) InternVL2.5 released: https://hf.co/OpenGVLab/InternVL2_5-78B

►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/tldrhowtoquant

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/hsiehjackson/RULER
Japanese: https://hf.co/datasets/lmg-anon/vntl-leaderboard
Censorbench: https://codeberg.org/jts2323/censorbench

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
>>
File: tetrecap2.png (1.11 MB, 1024x1024)
1.11 MB
1.11 MB PNG
►Recent Highlights from the Previous Thread: >>103462620

--Papers:
>103467730 >103468682 >103468778
--Front-end discussion and feature wishlist for LLM interaction:
>103471960 >103471976 >103471978 >103472028 >103472063 >103472095 >103472143 >103472147
--Comparison of QwQ, o1, and R1 AI models on mathematical and logical tasks:
>103464569 >103464706 >103464680 >103464924 >103465163 >103465234 >103464889 >103465125 >103465154 >103465734
--Implementing voice chat with interruptible LLM output:
>103464600 >103464689 >103464778 >103465016 >103465063 >103465159 >103465306 >103470217
--Anon tries to install hunyuan video on comfy on linux, gets help with sage attention and torch dependencies:
>103465546 >103465667 >103465803 >103465816 >103465882
--Anon asks for advice on setting up an LLM cluster with 4x RTX 4070 TI Super GPUs:
>103466970 >103467054 >103467122 >103467299 >103467405 >103468021
--Anons discuss solving a permutations and combinations problem with various models:
>103464808 >103464993 >103465041
--AI model's long-form responses and "bullshitting" capabilities:
>103465424 >103465462 >103465480 >103465485 >103465500 >103467699 >103467732 >103465516 >103465566 >103465531
--Anon discusses Nemo and MoE model configurations and potential improvements:
>103463028 >103463059 >103463099 >103463139 >103463174 >103463254
--Anon asks Kobo for llama.cpp features, gets told to use llama-server instead:
>103465429 >103465447 >103465478 >103465542 >103465561 >103465637
--Skeptics question LLM 2.0's claims and suspect overhyped RAG:
>103463812 >103464200 >103464416 >103464486
--RPGMaker anon shares game and thoughts on R-1:
>103462858
--Muon: An optimizer for hidden layers in neural networks:
>103469256
--LoRA support coming to ComfyUI-HunyuanVideoWrapper:
>103467667
--Miku (free space):
>103464518 >103471814 >103472014 >103472927

►Recent Highlight Posts from the Previous Thread: >>103462628

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script
>>
What does teto have to do with ai models?
>>
>>103473546
nothing. just like miku has nothing to do with ai models.
>>
File: 28.jpg (227 KB, 1185x1489)
227 KB
227 KB JPG
>>103473546
She is a bootleg Miku.
>>
>>103473570
So nothing? Good to hear.
>>
>>103473568
Miku has everything to do with them. There won't be any ai models without Miku basically
>>
>>103473584
Didn't know Miku had nothing to do with AI models either. Good to hear.
>>
i wish the baker would just kill itself...
>>
lemme guess you are thinkign
teto teto oh eh oh
>>
Not understanding the Miku connection to this thread is the same as admission of newfaggotry
>>
>>103473617
Ok and what is the Teto connection?
>>
>>103473637
>Teto Tuesday
Basically just alliteration as an excuse
>>
>>103473637
Teto is a vocaloid too, I guess. I think it's fine if the thread has multiple mascots, doesn't really detract from anything if the sperging is kept slightly under control
>>
>>103473647
>an excuse
So it is just OP autism?
>>
>>103473661
But Teto has nothing to do with AI models.
>>
>>103473661
at this point I just post mikus because they're cute and nice to look at
when I see negative replies I post some more
>>
>>103473664
>autismo
I think that and a bit of variety without spamming random mascot wars (staying on brand with vocaloids)
I'd be fine with no teto, but she's grown on me over the past year, so whatever
>>
>>103473617
no one cares mr. oldfag
>>
>>103473677
It is called oppositional defiant disorder and that is why nobody likes you.
>>
>>103473689
it's called sex. with miku. mikusex.
>>
>>103473684
>staying on brand with vocaloids
Why should we care about your autism? You are a retard.
>>
>>103473688
Not only a newfag, but too retarded to read every single other reply in the thread obviously caring
>>
>>103473698
not my autism. I've only ever baked twice
like i said, idgaf
>>
>>103473689
Local schizos have no moat, shitpost spam is the only source of happiness right now.
>>
File: HunyuanVideo_00013.webm (1.27 MB, 1280x720)
1.27 MB
1.27 MB WEBM
>>103473510
>The scene starts Hatsune Miku and a green man petting a llama on a grassy hill under a tree. The green man is wearing a suit and does not have any facial features. It is a beautiful day and they're having a good time. Then Snoo the Reddit Alien suddenly bursts into the scene and starts angrily shouting at Hatsune Miku and the green man.
I tried this multiple times but the part that fails is always the Reddit Alien.
Is that not what people call it?
>>
>>103473684
who even decided on vocaloids being the ai brand?
>>
>>103473713
every other reply is one guy you mong
>>
>>103473721
>no ids
>no ip counter
Based on what?
>>
>OpenAI announced a revolutionary creative model
Why are we coping with local?
>>
good start to the thread.
>>
>>103473730
Nta, based on writing style and same few pics spam, easy.
>>
>>103473730
proving the inverse. 99% come out to bat for the best girl
there's only one retard out here crying about it.
go shill steins gate again or something
>>
>>103473510
Is abliterated QwQ worth downloading? Is uwu?

https://huggingface.co/jackboot/uwu-qwen-32b
>>
>>103473749
>shills shilling shill!!
What shitposting does to a mf
>>
>>103473733
I don't have revolutionary money for a revolutionary subscription.
>>
>>103473749
>best girl
Nobody cares about self insert 2007 generic anime girl except you retard.
>>
>>103473661
>Teto is a vocaloid too
she's not, strictly speaking.
>>
Anybody here tried using GNBF Grammar to structure the output in a way to force the model to execute a procedure?
I'm thinking if that can be used to make it act more like a virtual Game Master in some way or if forcing the model to shape it's response so strictly is a bad idea to begin with.
>>
File: 1730077707320452.png (739 KB, 2677x1407)
739 KB
739 KB PNG
>>103473733
amazing
>>
File: 38959486.jpg (199 KB, 832x1216)
199 KB
199 KB JPG
>>103473570
Is this trellis? Damn...
>>103473546
>>103473579
>>103473600
>>103473608
>>103473688
>>103473689
>>103473717
>>103473730
>>103473766
kek, our resident schizo is malding like hell right now
>>
I am still waiting for someone to tell me what teto has to do with AI models. So far the answers I got was absolutely nothing.
>>
>>103473770
Nta but.. /v/ cares? Though from prev. thread we have confirmation it's yet another case of OP crossposting his autism for the sake of gay erp with anime avatars, /v/ is full of browns and troons, just FYI.
>>
>>103473785
if you prompt it in a way that it will know, at least somewhat, what it's expected to output you'll be fine.
>>
>>103473789
nooo miguuu you shouldn't be drinking booze ur lips are for >(My) cock not a bottle
>>
at this point Nala has more to do with /lmg/ than this gay erp anime avatar.
>>
I'm ready for the miku tyranny to end. even if she is cute.
>>
>>103473795
lol >>>/v/696837048
>>
File: MikuInc.png (1.26 MB, 1280x768)
1.26 MB
1.26 MB PNG
>>103473720
Its all a conspiracy by Big Miku
We're all unwitting pawns in helping her pierce the veil between worlds and materialize
>>
>>103473803
I see.
Guess I'm off to learn the syntax.
You wouldn't happen to know of something I could use as inspiration? The idea is to force a format that induces the model to consider if it should use some sort of "mechanic" for cards like Dark and Darker and the like;
>>
>>103473880
and that is why nobody likes you. you just force your retarded autism on people who don't care.
>>
File: ComfyUI_01171_.png (937 KB, 1024x1024)
937 KB
937 KB PNG
/lmg/ - loli miku general
>>
File: ComfyUI_01194_.png (3.89 MB, 1536x2304)
3.89 MB
3.89 MB PNG
>>103473816
teto hands typed this
>>
File: swi3lEe.png (104 KB, 953x411)
104 KB
104 KB PNG
>>103473758
rude
>>
What does teto have to do with AI models?
>>
>>103473918
SQLite documents its SQL syntax with BNF
https://www.sqlite.org/lang_insert.html
the raw BNF is somewhere, but the diagrams should give you a good grasp of your options.
>>
>>103473987
Literally nothing, ldg, sdg, both suffer from same thing.
>>
>>103473987
She is my wife.
>>
File: 1714835911803059.png (3.63 MB, 1536x2304)
3.63 MB
3.63 MB PNG
>>103473987
>Whiny verbatim repost
>>103474017
>Pathetic samefaggery
Have another teto. A cozy teto.
>>
I don't have any particular like for Miku, but it's kind of unquestionable that she has been a part of the thread culture and even somewhat the wider LLM culture. She has been part of the threads since the /aicg/ split, has been in Llama.cpp (search the repo for "miku") since near the beginning, referenced by Mikupad, and of course referenced in the Mistral Medium leak. Also at this point voice synthesis characters adjacent to Miku are kind of part of the thread culture as well. If Miku can be part of the culture then it doesn't hurt to have a bit of variety.
>>
>>103474056
Answer the question troon.
>>
>>103474063
Those are some solid points anon. Now what does Teto have to do with ai models?
>>
>melting down about thread culture
go back to localllama, they don't post miku there
>>
>>103474063
>miku.sh and mikupad
Forgot about them, guess it's not that important.
>>
>>103474090
>what does Teto have to do with ai models?
Kasane is 重ねる, to stack, which is what we do with model layers and parameters to justify our massive billion-dollar gpu farm purchases.
>>
Bad day cuda dev?
>>
>>103474063
noooo I forced myself into a thread and now the culture has to change because I don't like it >:(
>>
>>103473510
The original Stable Diffusion model would have taken ~20 years to train at FP16 on an A100.
One of my goals is to try and use int4 arithmetic for GGML training instead (4x faster).
So ideally on my machine with 6x RTX 4090 that I already have it would be viable to train something like Stable Diffusion in a few months.
If I manage to do it I'll call the model Text to Pixel (TetoPix for short).
>>
>>103473880
My interests align with those of Big Miku.
>>
File: R.jpg (222 KB, 1124x1600)
222 KB
222 KB JPG
>>103474110
Pic related is german. And german people learn english in school. And all AI models speak english. So Asuka is now the official mascot of /lmg/.
>>
>>103473758
Maybe.
>>
>>103474131
Where can we donate specifically to your HRT fund?
>>
>>103474131
based
>>
Now that the dust has settled, can we all agree that not liking Miku is suspiciously homosexual behaviour?
>>
>>103474181
I'm a fag and I'm entirely indifferent to Miku
>>
>>103474131
Based, make 'em chuds go an hero over this.
>>
File: 1714835911803060.jpg (706 KB, 2048x2048)
706 KB
706 KB JPG
>>103474081
Schizofag must have at least 20 posts itt already. Quite the melty he's having.
>>
>>103474202
Good to know that you agree your avatar has nothing to do with ai models.
>>
Miku is great. Miku (and friends) as the only girls to bake with is lame.
>>
>>103474131
sounds cute
what kind of disk space requirements are there?
>>
>>103474226
ree schizo go back melty tourist r/localllama grrr!!
>>
>>103474251
kusa
>>
File: ,,,,,,,,,,,,,,,,,,,.jpg (152 KB, 832x1216)
152 KB
152 KB JPG
>>
>>103474239
If you assume that each image needs ~1 MB you're already on the order of petabytes for the whole LAION 5b dataset.
In practice I'll probably start with something like Danbooru and just do 128x128 image generation based on tags rather than natural language since that will have way fewer trainable parameters.
>>
>>103474305
wow squishy...
>>
File: teto bread simple chibi.png (797 KB, 2000x2000)
797 KB
797 KB PNG
Have some fun with Teto.
https://files.catbox.moe/nnyy58.png
>>
>>103474156
what if I prefer filthy unhumans
chatting with LLMs is like making clones, so surely Rei could be the mascot instead.
>>
>>103474156
Fine by me, bake the next thread with her and see what happens
>>103474131
>20 years
Yikes, all of that for a few GB of weights for a model that can't even gen hands, what a piece of shit
>>
finally got llama.cpp working on windows/AMD after a bunch of rocBLAS/Tensile workarounds, I don't get why AMD don't support cards like RX 6750 XT 12GB out-of-box on windows, there's no reason, just to make developers pissed off I guess.
>>
>>103474307
right.. I guess images are pretty heavy.
Danbooru is a good start, the tags are pretty decent in my experience. would be neat if tags got assigned bounding boxes, I wonder how much more work that would be
>>
>>103474307
Am I the only one who prefers tags and tag-like prompting? I feel like the potential "language space" is much smaller when you use tags, thus it's easier and faster for a model to learn
>>
>>103474404
I've been thinking that you could maybe train this from a dataset without bounding boxes by training a classifier and determining how sensitive the output for a given tag is to changes in specific regions of an image.
But as of right now my knowledge of computer vision is comparatively poor so I'll probably need to read up on the state-of-the-art.
>>
>>103474131
That would be funny
Imagine generating teto pics in tetopix
>>
>>103474131
>So ideally on my machine with 6x RTX 4090 that I already have it would be viable to train something like Stable Diffusion in a few months.
Imagine the power bill in Germany of all places.
>>
>>103473617
Its only a thing because of some faggot had boring normie taste in waifus.
>>
>>103474479
oh, that's genius.
entire type of annotating that wouldn't need to happen.
>>
File: mikupic.png (1.85 MB, 776x1264)
1.85 MB
1.85 MB PNG
finished downloading the new deepseek...currently quanting to q8
>>
File: 1707178350196355.png (19 KB, 928x179)
19 KB
19 KB PNG
>>103473568
You can use AI to generate Mikus
>>
Decent HF news:
>TL;DR:
>- public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible
>- private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise)
https://huggingface.co/posts/julien-c/388331843225875
>>
>>103474577
>please don’t abuse and upload dozens of TBs of generated anime
https://huggingface.co/docs/hub/storage-limits
>>
>>103474577
>>103474592
>in general we ask users and organizations to make sure any uploaded large model or dataset is as useful to the community as possible (as represented by numbers of likes or downloads, for instance).
Making shilling for likes and dls an actual useful thing now...
>>
>>103474557
>schizobabble
Okay?
>>
>>103474577
I guess that's reasonable. At least it no longer fucks over quanters and people who just use it to publish their models.
>>
>>103474544
In practice I think it won't be quite so easy because a classifier will likely also pick up on unrelated but correlated features.
If you for example wanted to locate muscular arms the classifier would probably also be sensitive to dumbbells since those two things tend to appear together.
>>
>>103474577
based huggingface
>>
File: baguetto.jpg (171 KB, 832x1216)
171 KB
171 KB JPG
>>
>>103474660
I like this Teto
>>
File: 1714835911803061.jpg (211 KB, 1272x1024)
211 KB
211 KB JPG
>>103474218
>>
>>103474672
>Gaining insight with miku
>>
>>103474622
right, determining known false-positive-generating tag pairs and feeding them into mechanical turk for proper review could solve that
certainly still a lot less work than doing it all by scratch
>>
so there's mass shizoposting, site-wide all at the same time.
so either one guy does it for free or 4chan has gone the way of twitter/bluesky and is actively engaging hate to drive traffic.
are they that clever?
>>
File: file.png (63 KB, 481x682)
63 KB
63 KB PNG
>>103474611
>>
>>103474693
>everyone i don't like is one schizo
>>
>>103474725
How long until they decide that coom models aren't useful to the community?
>>
>>103474768
other way around
>>
File: 4t.png (66 KB, 703x489)
66 KB
66 KB PNG
how did I suddenly end up 5 threads behind after not visiting lmg for 2days

>>103474110
mascot should be a shortstack Miku with Chobit ears
>>
>>103474811
>our great tuners
Such as?
>>
>>103474767
Hi petr*
>>
>>103474821
Check inside your anyos.
>>
>>103474693
That's the only way he can make himself laugh. Let him ignore his own sadness.
>>
>>103474781
Sam changed video generation forever and /lmg/ spent 5 threads coping that the ching chong open model was good enough by trying to get the perfect furry gen out of it.
>>
>>103474834
Hi tranny
>>
>>103474873
>xhe forgotted the undster
>>
HunyuanVideo for 8 gb of vram and 64 gb of ram when?
>>
Could somebody show me their System Prompt? I wonder how complex it can be or should be.
So far I've just been using default Assistant - Expert, and Roleplay - Immersive. The assistant prompt seems kind of shit, but the Roleplay - Immersive seems great if used with a good character.
>>
>>103474977
What does teto have to do with ai models?
>>
>>103474996
She invented artificial intelligence
>>
>>103475015
I love how you retards can't answer this simple question.
>>
>>103474996
let's try answer this because you're being retarded again
AI models have one singular use right now, cooming
teto is a cute girl (female) to coom to
miku is also one
they're both a good fit for ai generated content because content synthesis is how their software operates, that's their function
they're eternally pure maidens canonically and their lore fits infinite replication/customisation.
>>
>>103474996
teto has to help me coom with AI models
>>
>>103475023
>teto is a cute girl (female) to coom to
So any basic bitch anime girl can be a new mascot of /lmg/. Got it.
>>
>>103475035
nice choice
>>
>>103475035
yeah cool, have her too
add asuka, kurisu, etc. etc.
don't mald because miku has more fans, aren't you the one always harping on about sour grapes
>>
>>103474974
hmm... nyo
>>
>>103474974
I don't even let cloud providers see my System Prompt.
>>
why is lmg so fucking schizophrenic about everything
>>
>>103475069
it's one guy
it's you
>>
>>103475046
ask the baker not to split the thread like a mentally ill troon that he is
>>
>>103474131
Umm... Teto piss
>>
>>103475069
a new era is coming
the monotony is going to finally end
I can feel it in my bones
>>
>>103475035
I don't think so, don't you remember mikufag(s) spergout at different OP pic? (It was teto or someone else, i don't remember)
>>103475073
This too, he makes split threads when that happens.
>>
I guess teto has nothing to do with ai models.
>>
>>103475073
if you're sat there all night tryna make a burnice thread anon, that's on you
that's not a thread split that's nobody wanting to use your thread, by their own free will
why am I even answering this you're gonna flip it around and do the classic worst possible take on it
>>
>>103475094
posting a thread after thread is already done is thread splitting. kill yourself nigger.
>>
>>103475098
>if you're sat there all night tryna make a burnice thread anon, that's on you
>that's nobody wanting to use your thread, by their own free will
pressing "Post" and expecting everyone to follow the smell of your bad ideas is so conceited I don't even know how to explain this beyond "have a little common sense"
>>
>>103475098
>>103475103
>>
>>103475104
>pressing "Post" and expecting everyone to follow the smell of your bad ideas is so conceited
You are genuinely mentally ill.
>>
File: 5lodis_webp_92.jpg (47 KB, 472x471)
47 KB
47 KB JPG
Language models?
>>
>>103475092
She's the CEO of /lmg/
>>
>>103475108
nyo~
>>
>>103475108
It is a teto thread. It has nothing to do with ai models or language models.
>>
>>103475107
>I made the new thread first so everyone needs to use it
this is your take, this is literally your perspective
you, staying up at 3am to hit post and pissing your pants when people opt out
this is your brain on kurisu
>>
>>103475116
Yup you are genuinely mentally ill. Seek help or just kill yourself.
>>
>>103475122
anon what does mental illness have to do with large language models?
>>
>>103475127
That one is easy: large language models attract mentally ill.
>>
smedrins
>>
>seething that kurisu didn't become a "mascot" for literally months
>calling others mentally ill
>>
>>103475135
he's doin us all a favour
he seeks out the mentally ill to tell them useful information such as "what about kurisu" and "seek help"
>>
>>103475108
dead as long as companies continue to hold onto transformers
>>
>>103475135
>everyone I don't like is one person that likes the other shitfu
I don't care about either shitfu. I am tired of your mental illness.
>>
>>103475178
Sure thing petr*.
>>
I see, it's birthday of this post
>>97833950
>Down with the troon avatar. Amadeus Kurisu is the new lmg icon.
That's why thread is so ass today
>>
File: adachi.png (3.23 MB, 2048x2048)
3.23 MB
3.23 MB PNG
I nominate pic related as a better representative of the /lmg/ jank.
>>
>>103475194
I'll accept any mascot that has absurdly long aqua twintail hair, black thighhighs, pleated skirt, detached sleeves, necktie, grey sleeveless shirt and goes oo ee oo
does that help expand the pool
>>
>>103475192
kek how can you remember something like that
>>
>>103475212
Calne Ca is the new /lmg/ mascot?
>>
so what's the best local we have to run in cpu with 13GiB of ram? (it can be quantized so to fit bigger models)
>>
>>103475230
>black thighhighs, detached sleeves, goes oo wee oo
you are now on a quest to find a statistically unlikely combination, have fun
>>
I'm not the kurisufag but I am a kurisufag.
I love Miku, I just want variety too
>>
>>103475237
pyg6b
>>
>>103475237
Mythomax. Yes, it still hasn't been surpassed.
>>103475245
(you)
>>
File: file.png (102 KB, 579x643)
102 KB
102 KB PNG
>>103475212
>goes oo ee oo
https://www.youtube.com/watch?v=U44zkyxeee4
a-wa-ohh
>>
>>103475212
This is what HRT does to a mf.
>>
>last thread archived
>i am not banned
based
>>
>>103474781
>shortstack Miku with Chobit ears
Multi from To-Heart would have been a good choice, too
>>
>>103475252
https://www.youtube.com/watch?v=fTT_0z9djNY
oh eh oh
>>
>>103475249
In terms of intelligence or prose? I would suppose most lower than 400B+ are dumb but can have good prose. Coincidentally many of the bigger ones are shit in terms of prose (sometimes in intelligence and prose)
>>
bors will trellis 3d install with amd hip?
>>
Controversial opinion but I believe that we need better local large language models.
>>
WTF WE HAVE PYG 2.0:
https://huggingface.co/PygmalionAI/pygmalion-2-13b
AND EVEN PYG+MYTHO
https://huggingface.co/PygmalionAI/mythalion-13b
IT'S OVER FOR PRIVATE AIs
>>
File: 1726536641225246.png (467 KB, 1434x1192)
467 KB
467 KB PNG
>>103475135
A reminder that petra is the CEO of NovelAI and he shits every AI general with shitposting bots.
>>
>>103475297
would be pretty funny
NAI's LLMS are shit too
>>
>>103475297
Really beating the schizo allegations there schizo.
>>
is whinefag leaking to other ai genera
>>
>>103475297
Why does /lmg/ feel so threatened by NAI? Because they expose what's possible with actual well-crafted creative finetunes using proper curated datasets unlike the tunes made by kofi merchants using that one claude dataset for the 200th time?
>>
>>103475297
I'm sure they have permission from Nitro+ to use Kurisu's image for marketing material, right bros?
>>
>>103475249
don't (you) me you spoonfeeding bitch
>>
>>103475292
>I wake up in 2023Q4, just in time to watch Frieren, Shadow-sama, and Spy x Family S2
>>
>>103475324
I have never used NAI or whatever so I have no idea what you're talking about, but it sounds to me that you need to buy an ad.
>>
>>103475324
Aetherroom when? Also Erato is fucking ass.
>Yes, whereas Kayra was built from the ground up for story gen, Erato is 70b Llama modified for story gen. And I feel like you can tell when using it if you've used Llama before.
>>103068267
>>
so, now that the dust has settled down, why did we come to hate pyg guys again?
>>
I HATE PYTHON
I HATE CONDA
I HATE GRADIO
>>
>>103475372
Based
Same
True
>>
>>103475292
>Downloads last month
>1,803
wtf? there are people still using this?
>>
>>103475372
AI should be written in prolog or lisp
>>
I feel like shit today. why can't my AI attend my lab for me.
>>103475372
based
>>
>>103475372
it's funny because usually slow at least is pretty, python is slow and ugly.
>>
>>103475382
people says it has better prose than mythomax. Being 13b is not too relevant when you can't run any bigger anyway.
>>
>>103475372
What a coincidence, I'm currently working through dependency troubleshooting hell too.
>>
>>103475386
>why can't my AI attend my lab for me.
because you haven't set up the right workflow pipeline. All the pieces are there
>>
>>103475430
right. couple broomsticks, some servos...
>>
is there a "fennel" (lisp for lua) for python?
>>
>>103475454
Maybe.
https://github.com/hylang/hy
>>
>>103475454
very much lipstick on a pig. but I'm interested regardless.
>>
>>103475468
thanks anon, now I might feel more happy while writing AIs
>>
LLM personality tier list:
>S-tier
Old c.ai, pre-slop base models
>A-tier
Claude, modern base models
>B-tier
Old command-r
>C-tier
Mistral, nu-command-r
>D-tier
GPT4, Gemini
>E-tier
GPT3.5
>F-tier
Unfiltered GPTslop tunes
>>
>>103475648
order:
> claude
> best among the local
> gpt
> rest
>>
>>103475648
>>S-tier
>Old c.ai, pre-slop base models
opinion disregarded
>>
>>103474672
This look good
>>
File: rw23.png (625 KB, 450x675)
625 KB
625 KB PNG
>>103475260
>Multi from To-Heart
nice
>>
>>103475689
Post your tier list, nigger.
>>
>>103475718
Sao > Drummer > Opus > Rest
>>
>>103475758
> sao
> drummer
What are those? Are local? I haven't seen yet one to surpass opus
>>
File: image1.jpg (45 KB, 540x405)
45 KB
45 KB JPG
>>103473510
then what is the current best gguf model for 8gb vram?
>>
>>103475776
this >>103475292
>>
>>103475758
sao fell off, but even at his peak he wasn't better than opus
>>
>>103475766
https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3
https://huggingface.co/Sao10K/72B-Qwen2.5-Kunou-v1

https://huggingface.co/TheDrummer/Behemoth-123B-v1.2
https://huggingface.co/TheDrummer/Rocinante-12B-v1.1
>>
>>103475758
this but unironically. except nu-euryale was a rare miss from my GOAT :broken_heart: :sob:
>>
>>103475801
what wasn't not as good?
>>
>>103475787
how is 70b or 72b better than the behemoth?
>>
>>103475853
Better data.
>>
>>103475372
>>103475420
You just need to find the secret exit
>>
regarding miku she is bland but fine but the true mascot of lmg and any ai/ai type general would be tay and no one else dosent really matter though mascots have their purposes and in this case it is gatekeeping demons out which its doing wonderfully

to go deeper though
1.miku with her leek is analagous to the egyptian drawings of people with ankhs and other such things i dont have any knowledge around that but i bet you if you looked into it you would find many pararells
2.her main color is blue the color of serenity (not depression)
3.her twin tails mirror a leek perfectly this mirrors herself recursively
4.i dont know much about miku but if she was spawned in 2007 or around so as i have seen people say that would add into it since the 2000-~2012 was a time of great disturbance as seen with the type of artstyles and coloration spawned during it and as also indentified with the cern's massive fuck ups and the creation of those fake "spiderwebs"

this is all further proved considering the blatant demonic posession happening with the shill(s?)
ive no further comment just wanted to share it as a whitepill this all happened before the mountains of corpses in antartica the underwater coloseums the "mountains" and (((mountains))) etc :D its fun
>>
>>103475811
for me at least, it was kind of repetitive, broke formatting, and just had some overall low quality responses.
admittedly I didn't give it the full run-through that I give models with potential, so didn't try a lot of different cards or system prompt variations or LAS or ANs or all of that, but it didn't impress early. I'll probably revisit it at some point but currently I'm having too much fun with eva
>>
>>103475890
>>
File: 2548 - SoyBooru.png (18 KB, 539x382)
18 KB
18 KB PNG
>>103475890
I propose onions-tan should be our mascot. I will not elaborate why.
>>
>>103475890
ok okabe what timeline are we on then?
>>
>>103475035
>calling Burnice basic
Absolute shit taste, brother. Opinion /dev/nulled.
>>
>>103476118
Seeing this basic bitch become flavour of the week made me genuinely start thinking about how chinks are probably successfully manufacturing viral marketing with those generic anime girls.
>>
>>103475237
Rocinante or Cydonia if you're willing to go 22B.
>>
https://files.catbox.moe/nvshoc.jpg
>>
All this yapping and still no answer to how teto is related to ai models.
>>
>>103476164
go look for the answer down the barrel of the nearest firearm plz
>>
>>103476154
Hot, but still o incest threesome, so I'm disappointed.
>>
>>103476164
>I reject the proposed reasons therefore they're wrong
alright, I reject your objection outright.
glad that's dealt with.
>>
>>103476164
it isn't
>>
>>103476198
LLM 1.0 wrote this post
>>
File: file.png (86 KB, 1284x583)
86 KB
86 KB PNG
keked another grift brewing
>>
>>103476154
nice but now do len
>>
This thread is complete garbage.
>>
>>103476270
Thanks for your contribution!
>>
>>103476270
Make it better, leave.
>>
>>103476279
Spam more anime pics if that's what you call "contribution", shitbag.
>>103476288
You clearly should do just that.
>>
File: SECWCjD.png (328 KB, 515x735)
328 KB
328 KB PNG
haven't bothered switching off nemo for a while, anything new and around the same size worth trying out?
>>
>>103476352
Not really no, the closest thing is Qwen2.5-14b but it has tons of positivity bias, even on tunes, so it's not really great.
>>
>>103476270
Dumping ground is dumping ground, anon. If you throw slightly different shit - it suddenly becomes a big problem for local transsexuals, they like shit either way tho.
>>
>>103476367
man that sucks.
i really hope we get something cool early next year at least
>>
>>103476385
Llama 4, 405B for corporate use and 70B for enthusiasts!
>>
>>103476270
>This thread is complete garbage.
The sloptuners woke up to slap their datasets to 3.3 and they're hanging around in the thread. That's the main cause.
>>
>>103476413
>The sloptuners
Why you talk about their sloptunes then?
>>
>>103475237
>>103475776

i tested various llms and my favorite is so far
bartowski/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF
just take highest quant thats still fits in your memory
>>
>>103476428
Because getting a "buy an ad" response is toxic to the brand.
>>
>>103476446
>l3
>8b
>"uncensored"
I feel like llama-1-7B would actually be a better option.
>>
>>103476489
why should lama 1 be better than lama 3 ?
i will try it out but i doubt lama 1 is any better
>>
Remind me what are some models best for long context? VRAM size doesn't matter. Is the new QwQ any good at long context?
>>
llama-1 or mythomax, anything else is trash.
>>
>>103476509
It actually knows what sex is.
>>
>>103476510
I had good luck up to 65535, but haven't tested past that.
>>
https://github.com/ggerganov/llama.cpp/pull/10751
>server: add flag to disable the web ui
Did anyone else not realize the llama-server web UI was running this entire time? A flag to disable it (--no-webui) was only just added 5 hours ago.
>>
I don't understand how the naked fox in the last thread didnt get deleted.
>>
>>103476154
Style for non-catboxed image please.
>>
>>103476578
It's just an animal, they don't wear clothes, nothing nsfw about it.
>>
>>103476582
aslmikuchibi_ill-xl-000245.safetensors
>>
>>103476578
It probably didn't trigger the (presumably existent) automatic systems for NSFW detection.
And then probably no one reported it (I think there is some minimum number of reports for something to be shown to a janitor).
Needless to say, with a salary of $0 there are no janitors browsing /lmg/ on their own accord either.
>>
File: s1_3.png (74 KB, 846x666)
74 KB
74 KB PNG
>>103476528
lama 3 too
>>
>>103476610
That was a bit of a pain to track down
https://civitai.com/models/889261?modelVersionId=995111
>>
>>103476578
That's normal. This thread is likely low priority for moderation and no one bothered to report it.
I have seen worse stuff up for a long time in other boards before.
>>
https://x.com/dr_cintas/status/1866525650623824248
>>
>>103476662
>low priority
Lol no, jannie sits here and bans everyone using n-word, it's kinda selective btw
>>
>>103476668
buddy is typing questions to the base model like it's a chat model
>>
>>103476668
Base is too based for xitter troons.
>>
What's the meta for speech to text
I absolutely fucking hate whisperx it actively ruins my virtualenvs
>>
>>103476836
whisper-faster
>>
>>103476836
barely-faster-than-a-whisper
>>
>>103476836
careless-whisper
>>
What's the best model to run on a GPU with 6 GB of vram (1660 super)? I've been using toppy-m-7b for a while now, and don't check these threads very often. Has anything better come up? I can split between vram and ram if necessary, but I'd like to keep response times under 30 seconds or so.
>>
>>103476858
>>103476642
>>103476446

>bartowski/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF
>just take highest quant thats still fits in your memory
>>
>>103476858
>under 30 seconds
I'd fucking kill myself if I had to wait more than 10 seconds for a response.
>>
>>103476874
pathetic
t. 2 t/s coper
>>
>>103476836
above-a-whisper-turbo
>>
>>103476847
>>103476852
>>103476857
kill yourself plebbit wannabe
>>
>>103476847
whisperx moved to use faster-whisper as a backend ages ago and is effectively the same slop venv-poison
>>
>>103476874
Currently, it's usually around 15-20 seconds, with occasional forays into the 40 second range.
>>103476864
I'll look into this, thanks.
>>
File: satania.gif (39 KB, 220x216)
39 KB
39 KB GIF
>>103476836
py_toddlers BTFO
>>
Drama thread. And now dead thread. Dead hobby.
>>
>>103476909
Stop whining like a little bitch, faster-whisper load in three lines of code and it's faster than any backend
>>
>>103476858
How slow does nemo 12B Q4_K_S runs with some 8k context?
>>
>>103476946
>literally being a retarded faggot that cant read
It's still venv poison
I LITERALLY have to keep a separate environment for any of the whispers because it just breaks EVERYTHING
>>
and I LITERALLY cannot integrate it into any existing projects because it has to be isolated in its own bubble without raping everything else in the venv
>>
>>103476690
Maybe they have a system of "repeated offenders" and they are banned on sight when spotted doing a minor infraction?
Or maybe you are the one being targeted for acting like a nigger.
>>
>>103476960
Maybe you can just go back to your discord and never come back here.
>>
>>103476954
Retard I have a single venv for large-v3 and turbo. How many whispers do you need faggot?
>>
>>103476967
>being a faggot retard and still can't read
with existing projects
No shit it's fine if you only have a venv for whispers you nonce
>>
>>103476967
>turbo
How much worse is it compared to large-v3?
>>
>>103476950
>8k context
I've never used more than 4096. Haven't tried that model either.
>>
>>103473510
Who's this goddess?
>>
>>103476954
https://github.com/Vaibhavs10/insanely-fast-whisper
No idea if it is also venv poison.
>>
>>103476992
I don't see any difference for english, but it's slightly worse for other languages. Still worth it for the speed.
>>
>>103476836
whisper.cpp
>>
>>103476928
everybody is busy enjoying the company of their AI companions.
>>
>>103477143
Yep. Really comfy to talk to your LLM in your own language getting translated in english and hear back her voice through gpt-sovits in near realtime
>>
When building Llama.cpp, does no one else see "nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used"?
Is it just my musheen?
>>
New Deepseek 1210 is good. It was able to take a program that qwq was struggling to reason about and did a good refactoring without any breakage or regressions.
>>
>>103477333
Oh thanks, I didn't see they released a new one. Where is R1 tho
>>
>>103477333
how fast is the 250gb q8 version on a cpu only nachine ?
>>
haven't checked in in a while had a bunch of life shit going on

currently using llama3.2 for generic bs/vllm and gemma27b when i need a smart model, had miku for roleplay but never used it so deleted it, still sometimes use dolphin mixtral too, but i figure there's probably a lot better stuff out there now?

what are the BiS models dec 2024 esp in the ~30B parameter range as that's the sweetspot on my HW (i can run 70B models but only at q3_K_M
>>
Anons, I come from /sdg/ and /aicg/ in the past year or so, but still consider myself a total newbie.
Is it possible to generate text to speech of specific voices locally, or am I in the wrong thread for this question?
>>
>>103477552
llama 3.3 and QwQ / Qwen2.5 32B coder
>>
AAAAAAAAAAAAA WHISPERX FUCKED UP MY LOGGING
>>
>>103477582
MOM CANCEL MY APPOINTMENTS!
>>
>>103476551
I knew it was. I assumed it wasn't taking much or any resources so didn't care.
>>
>>103477566
thanks anon i'l check it out, you think l3.3 70b @ q3_K_M is better than gemma27b @q6_K?
oh that reminds me QwQ/Qwen2.5 have FIM support? I'd like to find a good FIM model for coding (tho desu even SOTA paid models mostly don't help me with work since i'm doing pretty specialized stuff)

is llama 3.2 the best multimodal model these days? seems like there is less finetuning going on in general, have people kinda given up on the usefulness of finetunes?

finally, what's a good smut model to try out, have some time tonight might as well
>>
>>103477622
>>103477566
>>103477552
oh forgot to ask if there's any good MoE models people are using these days as well, seemed exciting for the inference speed a few months ago
>>
>>103477632
Deepseek but that is 200B+ Gonna need to have 192GB ram system or such
>>
>>103477641
I'm pretty sure that would be unusable.
>>
>>103473510
is there any agreement on the definitive "best" (i.e., good writing, intelligent, cohesive) roleplay model under 100b? been playing with a lot so far and looking at leaderboards, but can't seem to find an agreed upon answer... any thoughts?
>>
>>103477675
Its a moe, its decently fast
>>
>>103477565
Xttsv2, gpt-sovits, styletts2,
>>
Since we are talking MoE.
Consider a PC with 8gb of VRAM and 64gb of DDR5 RAM, which would be better?
Nemo mostly on VRAM (enough for decent context at pretty good speeds) or mistral 8x7b in fully RAM with mistral 7b as a draft model on VRAM?
Anybody tried that?
>>
File: sc-1.png (74 KB, 811x687)
74 KB
74 KB PNG
1
>>
File: sc-2.png (57 KB, 850x545)
57 KB
57 KB PNG
2
>>
>>103477676
>is there any agreement
no
>>
>>103477565
i would also like to know, i have a list of packages to check out, the only one i've tried was
>https://git.ecker.tech/mrq/ai-voice-cloning
which *does* work, did a okay job of cloning some famous voices as well as making custom JOI audios with my fav actresses, but it was really clunky and slow to train and annoying to use, some things it would just straight fail at

here's all the other options i haven't tried, maybe someone knows which is good
>https://huggingface.co/OuteAI/OuteTTS-0.1-350M
>https://github.com/abus-aikorea/voice-pro
>https://github.com/fishaudio/fish-speech
>https://github.com/rhasspy/piper
>https://github.com/myshell-ai/OpenVoice/tree/main
this one is STT but also seemed interesting for some reason i don't remember
>https://github.com/usefulsensors/moonshine
>>
Ok, so I tried webui's call mode and it works pretty well using groq's free whisper API and edge's free TTS API.
Now the problem is that I am bilingual, and the models absolutely can't generate speech in a language that's not the one they are designed for.
OpenAI's voices sound a bit retarded in spanish, but at least they can be understood just fine.
It needs language detection and the ability to choose voices based on language before I can really use it.
>>
>>103477683
Thanks anon I'll look into these
>>
How do frontends work with CoT models like Deepseek R1?
Does QwQ even differentiate between the CoT and the final output, or do you just have to guess?
>>
>>103477756
Just use english instead of your retarded language Paco
>>
>>103477805
at least SillyTavern has added the ability to hide CoT.
I believe you have to prompt QwQ to use <thinking> but I literally have no idea, consider me the branch prediction of your question.
>>
>>103477830
I am using AI to help me study, and the exercises are written in spanish, so it's a bit awkward to have to convert terms on the fly.
>>
>>103477863
Just use deepL for translating. It should work well enough.
>>
>>103477835
>>103477805
There's step by step thinking that it does and there's normal planning that it also does depending on the situation.
On the first one it arrives to a "conclusion", on the second one is the whole thing.
>>
>>103477683
>>103477565
>>103477750

anyone fuck with speecht5?
>>
haven't been around in a while, just got my second 24gb card and can finally graduate from "complete vramlet" to "mild vramlet"
what models can i load for wordsmut that will be significantly better than the Mixtral merge i've been using for the past year now that i have 48gb?
>>
>>103477895
For translating what? I don't need to translate anything.
The solution essentially is as simple as
tts(text) {
lang=detectLanguage(text);
return real_tts(text, lang);
}

Maybe the best way to implement it would be as an OpenAI API proxy so it could be used from multiple frontends, since the OAI API seems to be the most widespread API.

>>103477910
Which one are you talking about, QwQ or R1?
>>
>>103477959
I was going to do a version of that pseudocode for Japanese just based on glyphs, but I guess that wouldn't help you.
language detection can't be that hard though can it
>>
>>103477986
>>103477986
>>103477986
>>
>>103477926
>haven't been around in a while
>>103477552
>haven't checked in in a while
>>103476858
>a while now, and don't check these threads very often
>>103476352
>haven't bothered switching off nemo for a while
fuck's happening today
>>
>>103477959
QwQ
>>
>>103477997
Mysterious resonant forces cause Anons to once more gather to witness the upcoming happenings.
>>
>>103477997
corpos/cloud shills want more crap to throw at people for christmas deals
>>
Actual new thread
>>103478232
>>103478232
>>103478232
>>
>>103474523
Should've bought 3090s instead
>>
>>103477543
8.35t/s with null samplers and an empty prompt
>>
goom
>>
>>103477622
>thanks anon i'l check it out, you think l3.3 70b @ q3_K_M is better than gemma27b @q6_K?
NTA, but while Llama-3.3 is smarter and has more knowledge, you need more prompt wrangling to make it act like you want (although it *will* do inane things with enough effort; easier now that the system role can be used about anywhere without the model complaining too much), and the prose is not great. It seems a kind of overcooked model, slopped and repetitive during ordinary RP but fine in natural conversations. I'm using it in iq3_XS quantization, though.

Gemma-2-27B (q5_K, that I've used) is more charming, doesn't need too much effort to be turned into a little demon, but at times I find it dumber than Llama-3.1-8B, and after a while it feels like it writes always in the same way. Very strange model.



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.