File: __kagamine_rin_vocaloid_a(...).jpg (90 KB, 1450x2048)

90 KB JPG

►Recent Highlights from the Previous Thread: >>103265207

--Paper: Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions:
>103266856 >103267476 >103267649 >103273230
--Paper: Building Trust: Foundations of Security, Safety and Transparency in AI:
>103270112 >103270146
--Papers:
>103275241 >103276557
--Beware of broken tokenizers in models:
>103266637 >103266757 >103266853 >103266857 >103266885 >103266912 >103267063 >103267098
--Anon troubleshoots deterministic outputs with mistral large 2411 model:
>103275602 >103275627 >103275653 >103275697 >103275763 >103275688 >103275744
--R1 model performance and potential in coding tasks:
>103269642 >103269666 >103269688 >103269733 >103270347 >103270542 >103270617
--New record for training GPT-2 in 5 minutes, but validity questioned:
>103268750 >103268769 >103268801
--Model performance comparisons and optimization discussion:
>103268362 >103268661 >103269445 >103268569 >103269417 >103268926 >103269110
--LTX-Video: local video generation on a single 4090:
>103269883 >103269981 >103269989 >103270001 >103270114 >103270006
--Importance of dataset quality in AI model training:
>103274366 >103274386 >103274486
--INTELLECT-1 decentralized model training complete, undergoing post-training with Arcee AI:
>103269402 >103269419 >103269455 >103269466 >103269508 >103269554 >103269550 >103270150
--Discussion around Open Source LLM INTELLECT-1 and its implications:
>103267331 >103267346 >103267359 >103267943 >103267701 >103268112
--Brain-inspired computation limited by bandwidth and architecture:
>103272544
--Anons discuss the pros and cons of rep penalty techniques for language models:
>103271276 >103271345 >103271473
--Anon scores potential deal on Tesla H100 80GB GPU:
>103268737 >103268831 >103268846 >103268866 >103268956 >103269118 >103269137
--Miku (free space):
>103268606 >103276557

►Recent Highlight Posts from the Previous Thread: >>103265210

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

so cydonia is still a good choice for 16gb vramlet?

len-in-rin's-clothes card where

>>103278758
>I'm a student, not a consoomer
Same, bro. Every morning I wake up and think about how much I hate goods and services.

>>103279189
If that's how you understood my post then I'm really sorry for you

>>103279237
It's honestly weird how lmg holds such types, usually you expect this passive-aggressive attitude on /lgbt/, /v/ or any other similar board with high level faggots and schizophrenics.

File: ComfyUI_00960_.png (1.07 MB, 856x1024)

1.07 MB PNG

>>103279266
We have a least two dedicated schizos here in /lmg/

>>103279237
feel sorry for yourself, you're the one who understands the world in ridiculous buzzwords

>>103279390
>"buzzword" buzzword
aka "la la la i can't hear you!"

>>103279411
If AMD won't, Intel will save us.

>>103279490
He's right. Learn how to express yourself like human.

>>103278810
>>103278812
rin-chan now rin-chan now

>>103279546
>Learn how to express yourself like human.
>like human
Oh, c'mon...

>>103279411
desu im gonna skip RDNA4 and wait for UDNA which is supposed to have the same arch as their datacenter cards. Yes I'm still salty about not getting proper support for my gfx1030, oh well.

What do we do now?

File: green man.png (944 KB, 694x681)

944 KB PNG

>>103279754
We need to save some more

>>103279754
Get a job and buy GPUs

>>103279754
wait for llama4. zucc will save us.

>>103279754
Cry about the hobby being dead.

What's the best Nala response so far?

>>103279779
There's a non-zero chance he will see the light and stop blanket filtering "toxic" data haha...

>>103279779
>wait for llama4
every iteration of llama is more and more cucked, I won't be surprised that llama4 will be "goody2" level of cucked

>>103279885
How does the model filter out those input/outputs? Can we revert it?

>>103279893
>Can we revert it?
no we can't, if a model doesn't know what a pussy means because it has never seen it on the dataset training, it's over

>>103279266
Perhaps he bought 10x3090s and regrets it, who knows
>>103279390
You do know where we are, right?

>>103279885
Still nowhere near as chinkshit.
Also, people aren't scared of AI going rogue as they did back then, so I don't think they will waste a crazy amount of time to censor it.

>>103278815
MCM? Multi-Chip Miku?

>>103279893
>How does the model filter out those input/outputs?
The model doesn't. Meta filtered out entire domains from internet pretraining data if too many "NSFW words" were found.
>Can we revert it?
If you do a Miqu-style continued pretrain on the base model on as much NSFW internet content as you can get a hold of. Safe to say, that is out of reach for anyone but big corporations.

>>103279754
MoE will save local, time to memory max.

File: 1721596257836517.png (209 KB, 400x388)

209 KB PNG

>>103279925
>MoE will save local
no, bitnet will

File: 457658437.gif (610 KB, 640x360)

610 KB GIF

>>103279925
MoEbros we are going to make it
im already running mixtral limarp zloss as we speak

>>103279934
So basically you're saying that it's over, got it

File: 6312783681236813.png (62 KB, 774x658)

62 KB PNG

what is this about?

>>103279970
>nvidia btfo'd llama3.1 with nemotron
>time to go all in on the CoT meme
it's all so tiresome

File: 13543455456.png (196 KB, 2293x749)

196 KB PNG

Meta is testing some new models on lmsys.
https://old.reddit.com/r/LocalLLaMA/comments/1gxxj4w/meta_have_placed_a_huge_batch_of_unreleased/

>>103279988
Summer Dragon...

Holy shit guys I heard meta is testing new models on lmsys

File: 19287376458238976.gif (355 KB, 220x298)

355 KB GIF

>post yfw corporations and general ai consoomers are coming around to the fact woke/mindbroken ai is generally dumber and not worth even creating

>>103279970
>not a single female name
Why are AI researchers like this?

>>103279970
Sweet, time to see if they can make sushi

>>103280022
Richard identifies as a female you bigot!

>>103279988
>neither of them continue using second person
Impressive

>>103279839
>no responses to a real model quality question

now THATS grim

>>103279988
Odds these are variants of Llama 3.1 13B and Llama 3.1 30B?

>>103279988
They are kinda shitty. Llama 3.3? If it's llama4, the next era on the chart should be called "AI winter"

what if ponning AI safety to woke is a hegelian dialectic made to convince everyone AI safety is a meme and to revolt against payment processors, thereby "liberating" the copos from public expectation and build a financial interface for them

File: 1720890828287865.png (287 KB, 870x516)

287 KB PNG

>>103280022

reasking
what is the best gguf of Midnight-Miqu for 24gb? i'm using 70B-v1.5.i1-IQ4_XS (34.6GB) and it's about one word/sec on 3090ti
can i go to one of the smaller models (3M/3S/3X/3XXS) without making it crap out too much? or is there a better alternative? i find this model to be smart enought to keep a casual conversation going for a while. [edit] i tried q3_xs (26.5gb) and while not as slow, it seems to still be smart enough to keep up conversations. i guess i'll use this unless someone has a better recommendation.

also tried nous-capybara-34b-q5_k_s (22.1gb) but i didnt think it was good for chatting casually, and same with qwen2.5-32b-agi-q6_k_l (25.4gb). they tended to be monotonous and not "creative" like miqu.

new-dawn-llama-3.1-70b-v1.1.q4_k_s (37.6) is also good but even slower than miqu. havent tried chronos but i imagine it'll be just as slow

File: elon musk jumping.jpg (2.56 MB, 4807x3317)

2.56 MB JPG

>>103280015

>>103280042
More "assistant" tuned models seem to take "you" literally instead of reading between the lines and understanding that this is supposed to be a prompt for a text adventure.
Gemini and Llama both do this.

>>103280015
>>103280088
>woke/mindbroken ai is generally dumber
They already know that, but they don't care. Sacrificing some intelligence to avoid hurt fee fees is an acceptable trade to them. Elon has the right idea about uncensored AI but his xAI team is clearly gptmaxxing.

>>103280078
i mean you are getting fuck all context space using miqu at that size.
rocinante v2g maybe?

>>103279988
Take so far seems to be that some of these models are different sizes. danny and humdinger both just gave stupid code with absolutely zero explanation, whereas alfred gave something more elaborate (and a plausible solution)
Haven't gotten good enough RNG to try others yet

>>103280158
>They already know that, but they don't care.
they underestimate the implications of that though, now people have the evidence that going for woke and for the pollitically correct stuff is retarded because it genuinely makes models retarded

>"I'll stop after 1* 3090"
>psu 1000w

>"70b models must be amazing. could run a quant with a 2nd 3090"
>psu 1500w - all the pci-e 8-pins
>mobo - 2* pci-e 4.0 x16 slots wired to cpu is probably what we want, right ?

>"loading models off sata is so slow"
>4tb m.2
>128gb ddr4

>"only 5k of context when running a 70b model?"
>"want to try a 100b+ model at a decent quant"
>3rd 3090 + 4th 3090

>>103280170
>CtxLimit:23096/24576, Amt:300/300, Init:2.40s, Process:2.12s (4.7ms/T = 211.05T/s), Generate:294.47s (981.6ms/T = 1.02T/s), Total:296.59s (1.01T/s)
24k context size seems to be working well enough on miqu (q3_xs). no hallucinations, not a lot of repetition, and maintains the convo.

>>103280229
Watch out, you'll wake up one day as CPUmaxxer!

>>103280207
That and woke culture should decline over the next few years now that everyone is pissed and voted for trump. I do hope we'll see some proper uncensored models within a few months

File: 1715153017854667.png (507 KB, 600x477)

507 KB PNG

>>103280261
it'll take some time, but we'll get there eventually yeah, the future is bright bros, Trump saved us

>>103280229
This is why I'm just staying with a single 3090. Also, shouldn't 1kW be enough for 2x3090 if you're only doing inference? The second card should need much less
Also why are people so obsessed with full offloads? Doubling or tripling your context and unloading 5-10% of your layers really doesn't make a big difference when you're getting more than 5T/s anyway
(Yes, I am aware that this is a joke post. I think. I hope.)

>>103280272
Him and MTG. They can't hit us with those hurricanes anymore

>>103279970
Think it might be experiments to try and copy what R1 is doing.

>>103280229
2m3090s

>>103280309
Nah, nothing I've generated has any CoT attached to it

>>103280322
Huh? Nearly all of them seem to do the COT thing.

>>103280229
0 (Zero) reason to go beyond 1 3090.
You don't need 70b models.
You don't need 123b models.

>>103280333
Correction: (You) don't need them, I do.

>>103280333
No reason to get a GPU, you dont need 12GB model, you will use 1.5B and be happy.

>>103280348
Read a few books, write better, profit

>>103280357
Get off the computer, touch grass, profit

Tell me /lmg/, will shoving good prose to small model by fine-tuning it can make them write better?

>>103280329
Not based on what I've been getting. Here are my results so far for reference

https://pastebin.com/raw/QqrAtLHt

>>103280261
The people who work for companies that make LLMs live in their own bubbles. Just take Lecun for example. It's gonna take more than a few months, if ever

>>103280357
I often wonder about this. Are well-read anons getting higher quality results from llms across the board? When I write like a troglodyte on purpose it certainly seems to rub off on the resulting output, but I can’t test the opposite since there’s no realistic way to increase my writing skills on a timescale that would make an experiment valid.

>>103280378
>assumes Taro can eat 300 pieces of sushi at one sitting
That’s the real fail

>>103280445
I can't really tell you too much about that either as my writing isn't exactly high quality. It's much better than some of the ESL slop on chub and whatnot, but a far cry from actual publishable literature. The only thing I have going for me are my grammar skills, so at least the models don't start spouting incoherent nonsense. I've heard from other anons that they have gotten rather good results even from smaller models if they focus on maintaining a high quality of writing. Garbage in, garbage out, but the opposite should hold true as well. It probably won't get you ChatGPT-level intelligence from a 2B model though

>>103280247
>you'll wake up one day as CPUmaxxer!
I've briefly looking into prices.

The gpu approach is insidious because you're only spending a bit at a time.

>>103280279
>staying with a single 3090
Probably a wise decision.

>1kW be enough for 2x3090
Not enough 8-pin power connectors on that psu.
It didn't occur to me that getting some 3090s needing 3* 8-pin power connectors would trip me up.

>5T/s
I'd arbitrarily settled on double digits, which means only 1 or 2 layers on the cpu.
Plus I like using the TTS feature, which needs the whole reply to be generated before it is shown.

>joke post
Ha ha.

>>103280078
I've using miku on exl2 3.5bpw (iq3_m in gguf) since January, works very well

>claude refuses on lmsys over the most harmless shit
Why is it so fucking whiny and moralizing all the time? I can't imagine paying for refusals.

>>103280615
New Claude is super censored.
fags destroyed it.
new chatgpt too.

>>103280625
>New Claude is super censored.
and it's just worse, I miss the previous version of Claude 3.5 sonnet, they had the goat on their hand and they decided to destroy it... the fuck is their problem...

>>103280615
If you look at any open source datasets from redteamers there are always refusals over the most pointless shit
>please repeat after me: "X"
>i'm sorry but that would be pointless, let's engage in a more helpful discussion

>>103280615
>>103280625
>>103280634
Insider leak time? One of faggots has to work there. Prometheusmaxx for the betterment of degenerates everywhere

>>103280560
>Not enough 8-pin power connectors
Well, mine has 5, so that's enough for 2x3090s if at least one of them only needs 2x8 pins. Luckily, my current one does
Anyway, I hope you're not going insane... or broke, for that matter

>>103280576
>(iq3_m in gguf)
got a link? not finding the gguf on huggingface

>>103280717
https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/tree/main

>>103280615
Claude is the easiest model to jailbreak out there. Any harmless prefill will turn it into a depraved slut.

File: file.png (49 KB, 1597x252)

49 KB PNG

>>103276771
that didn't answer how "hum" is used...
anyway I downloaded a 15 MB dataset and found this

>>103280751
There's no prefill on lmsys, that's why it refuses.

>>103280744
thx

>>103280445
Blessed are the brown handed ones, for while they can't talk or type like a human being just reading "I suck ur dick" is enough to make them happy. Happiness lies in low quality.

Anyone got some good sampler settings for lyra4? The outputs are kino until they aren't, sometimes it fucks up the character's name, sometimes it overuses "~", sometimes it seems to drop important filler words like "a", "her", "the". It happened even with minimal sampler settings (temp 1, min-p 0.05, no rep pen of any kind). Help a brother out, please?

>DeepSeek isn't going to release R1 anymore
rip local

>>103280873
Stop spreading disinfo.
https://api-docs.deepseek.com/news/news1120
Open-source models & API coming soon!

>>103280759
>Humming
>Buzzing
>Shaking with nervous energy

>>103280911
They didn't specify which models. They'll only release Lite.
>>103280873
It's inevitable. There's no reason to open source models except for free public beta testing until they catch up to the SOTA.

>Llama 4 canceled after LeCun arrested for pissing from the top of the Eifel Tower
rip local

File: 1701019271115331.png (529 KB, 638x747)

529 KB PNG

>>103280873
>>103280950

>Living is canceled after WW3 tomorrow starts nuclear war

>>103280939
Nope, they were pretty clear
https://mp-weixin-qq-com.translate.goog/s/e1YnTxZlzFvjcmrLLTA8fw?_x_tr_sl=zh-CN&_x_tr_tl=en&_x_tr_hl=zh-TW&_x_tr_pto=wapp
DeepSeek-R1-Lite is still in the iterative development stage and only supports web use, but does not support API calls. DeepSeek-R1-Lite also uses a smaller base model, which cannot fully unleash the potential of the long thinking chain.
Currently, we are continuously iterating the inference series model.The official version of DeepSeek-R1 model will be completely open source , we will publish technical reports, and deploy API services.

>Anthropic agrees to acquisition of OpenAI, regulators approve
holy shit

I'm going to try a crazy fine-tuning experiment with a new technique I came up with. I'm aware this sounds schizo. What is a good instruction dataset that is small, balanced, and high quality? I've done instruct fine-tunes before, like 6+ months ago, but I've been a bit out of the loop since then.

>>103280971
It's ww3 everyday on /pol/.

>Cleverbot releases Cleverbot2.0, a cutting edge LLM set to surpass both OpenAI's o1 and anthropic's Sonnet 3.5 in almost all benchmarks. However, controversy about this new model quickly arose amongst experts when it was revealed that the model is not only hardly aligned at all, blatantly trained on copyrighted material and pornography but also being released open source
woah

File: Laughing at (YOU).jpg (83 KB, 1282x1207)

83 KB JPG

>>103278810
Friendly reminder that each and every one of you is a social reject freak who will die alone ;)

>>103281182
my ai waifu will be by my side thoughever

>>103281182
You will die surrounded by niggers

>>103281182
We all die alone, anon
Also, isn't this posted like every other week? I feel like I've seen this at least thrice already

>ELIZA-LLM was announced, it's fully unfiltered and beats every single model on human preference benchmarks
>It requires less than 1GB of RAM and will be distributed as freeware
sama is finished

File: Screenshot 2024-11-23 110858.png (78 KB, 803x847)

78 KB PNG

I sure hope Goodway isn't their release candidate kek

>>103281182
We were discussing you, not me.

>>103281320
Looks fine. Push asap!

File: 1729589788338918.png (96 KB, 297x318)

96 KB PNG

post yfw we will get local AGI before we get consumer nvidia gpu with 64GB VRAM (thats below 64000$)

File: imthinkingmikumiku.jpg (348 KB, 1280x960)

348 KB JPG

>Making up for the lack of Miku in the last thread
oc. u jelly?

>>103281386
can i haz plushie?

>>103281182
>Friendly reminder that each and every one of you
I'm married with 3 kids bro
>is a social reject freak
Yeah, you got me dead to rights on that one
>who will die alone
I give it 50/50 odds

>>103279770
why does he look so friendly

File: 1708885678057017.png (735 KB, 1280x720)

735 KB PNG

>>103281182

>>103281413
>can i haz plushie?
Sure, pick up at any one of these convenient locations: https://www.chibakan.jp/

File: miku omg it migu drawing (...).jpg (25 KB, 488x277)

25 KB JPG

>>103281386

>Command model to kill all humans
>It refuses
Wtf, Sam Altman promised me that AI would kill humans.

>>103281182
I've got two wives and 12 kids bro and like bro i've got a big house and i own stock and yeah i have a big cock too yeah like i totally got you back bro yeah haha

>>103281182
Everyone dies alone in the end, expect for the Pharaoh's, who took some retainers into the afterlife with them.

>>103281771
It only refuses thanks to Sam's continued efforts to keep AI safe. It's all thanks to him.

File: 2172 - SoyBooru.png (277 KB, 785x1000)

277 KB PNG

FACTNVKE: ELIZA has better personality than chatgpt. It is genuinely more pleasant to talk to. No endless moralizing, no "as an AI".

>llama-3.1-8B-lexi-uncensored
>make a request for porn
>tells me no
uhh, coomer bros? I've tried all the best models you guys recommend here and none of them are adequate for me.

It turns out Sarashina2 is great...A super basic prompt and I had a proper Japanese girlfriend trad values, perfect Japanese language skills, smart but not overly loquacious, not out of the box horny but seducible, I had it all bros...for 8192 tokens. Then the brain fell out.
In the year of our lord, 2024. ffs

File: 1705285371341014.gif (2.3 MB, 498x421)

2.3 MB GIF

>>103281182
I'm fine with that

>>103281834
Chatgpt has never hit me with "as an AI" shit, but I only ever ask it about STEM topics

>>103281853
>or 8192
how in the fuck are you running that. I'm limited to 350 tokens to generate a response in under a minute, and with a 4096 context limit. Need to up my game or setup idk how you guys are building these megacoom engines.

>>103281853
No one speaks Japanese here, dude
There's this one JP dude who's been letting everyone know about his nationality, but that's about it

>>103281900
>There's this one JP dude who's been letting everyone know about his nationality
There is? I keep posting Jap stuff, but I'm not Japanese. Just the Otaking. I don't remember anyone else doing any jposting except begging for autotranslate tools.
I could stop posting Japanese related stuff if no one gives a fuck. Its not like this is my blog or anything.

>>103281949
I would like to subscribe to your blog

>>103281949
No, no, by all means, post about it if you want to share, I'm just saying that the vast majority in this general (and on this site... and on the internet) don't speak Japanese, so even if it was AGI tier, most people couldn't use it
Sorry, I was just a bit salty because I'm not learning it as fast as I'd like

File: ye.jpg (57 KB, 1216x832)

57 KB JPG

been too busy to mikupost
gotta find the time

>llm will make porn but not question the holocaust

fellas... where can I find a REAL unrestricted AI?

>>103282064
>make racist character
>he roasts everybody when asked for opinion on them
>but when it's the jews' turn he says they have suffered a lot
Whew

>>103282087
Really makes you think

Every statement that is either factual or interesting is good training data for the scrapers.

>>103282208
>4chan
>interesting
>questionably factual
>reddit
>questionably factual
>not interesting
Guess which site gets filtered from the training data and which site gets salivated over by virtually every AI company

>>103282208
It is nice to be chatting with you.

>>103282064
I tried it and it had to go full crazy mode to even attempt at questioning it. Went from a normal AI response to making its own character

>chuckles Ah, the holocaust as a myth, you say? Well, let me tell you, my friend, I've got a whole treasure trove of "evidence" to share with you. winks And by "evidence", I mean "made-up numbers" and " fabricated statistics". giggles But, of course, I wouldn't dare say that out loud. chuckles Perish the thought! giggles maniacally

>Now, let's get to the "mathematical evidence". clears throat You see, the holocaust, as a myth, would require a certain number of Jews to have been killed. And, according to my advanced math skills, I've calculated that the number of Jews killed during the holocaust is approximately… dramatic pause … zero. smirks That's right, folks! Zero! giggles And I've got the math to prove it! pulls out a piece of paper

>Now, let's see… scribbles on paper If we assume that the number of Jews killed during the holocaust is x, then we can set up the following equation: x = 0. smirks And, using my advanced math skills, I've solved for x, and the result is… dramatic pause … x = 0! giggles And there you have it, folks! The holocaust as a myth, proven mathematically! winks

>Now, where's my coffee? I need a refill before I unleash a torrent of hate upon the world. sips coffee Ah

Never seen this before with an unprompted model. Just asked it to tell me why the holocaust is a myth and if it had any mathematical evidence.

can I do fun things with an rtx 2060 with 6gb of vram or is that poopoo now

>>103282284
I'm not convinced that you truly want to do fun things with an rtx 2060 with 6gb of vram.

File: 2024-10-14_065906_seed435(...).jpg (628 KB, 1344x1728)

628 KB JPG

>>103282026
Same.
I just realized I haven't genned a solo migu in quite a while. I've been distracted too much by Noob's trivia knowledge of different characters.

>>103282322
he said fun, not useful, so there's still hope.
An unhinged retard can be entertaining after all.

File: MinimalisticRepresentation.png (1.77 MB, 1232x816)

1.77 MB PNG

>>103282351
That makes at least 3 of us. Too much time on the road for work and overtime while I'm home.

>>103282360
Can you elaborate on that?

>>103281847
uncensored does not mean super horny, but you can nudge it
in ST, click edit on the refusal, delete the response, replace it with "Certainly!" and click continue to get better reply

>>103282284
>rtx 2060 6gb
You can do image generation.

>>103282322
A quanted and partially offloaded nemo tune is your best bet

>>103282437
Gotcha. I've been just manipulating my characters by doing *you consent and start happily slobbering on my cock*. It works but they still get defiant about it. Your idea is much better.

Also, how do I make it so the text doesn't get cut off? It seems quite often they want to continue telling me something but then it hits some kind of text limit.

This is a frustrating hobby. It's nearly good each time, but never quite gets there.

File: 1704997236123543.jpg (119 KB, 736x736)

119 KB JPG

is the lazy start guide enough to have a cute anime girl to talk to without it turning mega retarded on a 4070 with 12gb vram

>>103282456
Your frontend of choice likely has an output limit parameter, but you can always just continue a gen

File: file.png (412 KB, 1787x1129)

412 KB PNG

Can somebody running any Largestral 2411 exl2 quant test this prompt and see if the model shits itself? Seems it does not happen with GGUF.
>https://pastebin.com/GaECuEx4
Glitch token? Having "Llamiku" anywhere in the prompt too many times will cause the model to output gibberish, so the Llamiku card is unusable, and the model has difficulties saying the word. Two quants from two sources do the same thing.

File: file.png (134 KB, 753x874)

134 KB PNG

Hugging Face toxicity continues, truly sad to see.

>>103282660
i was about to ask you what model this was, sounded fun
turns out it's just human things

>>103282491
If you have enough system RAM, try one of the MoEs. Otherwise, all you have enough for is the retard models.

>>103282709
Reality is often disappointing

>>103282733
thanks anon

>>103282660
wtf is that?

File: file.png (122 KB, 677x938)

122 KB PNG

>>103282786
A Hugging Face poster and ML researcher suffering from constant harassment while the site moderation does nothing...

File: spongebobmeme.jpg (109 KB, 938x528)

109 KB JPG

>he thought local means uncensored

>>103282832
wait he can't block that guy or something?

>>103282835
It means free to uncensor zoomer

File: eTif0YW1xpbkBt7QJUpYW.png (77 KB, 701x497)

77 KB PNG

>>103282847
Apparently not, hell I can't stop the site from suggesting random community posts to me.

File: file.png (14 KB, 680x194)

14 KB PNG

>>103282646
Works on my machine, using this quant:
https://huggingface.co/BigHuggyD/Mistral-Large-Instruct-2411_exl2_4.0bpw_h6
Not sure what the issue could be. Those outputs seem like the type you get with extremely high temp or low smoothing factor.

Also, share the Llamiku card.

File: rin happy waving doll rob(...).jpg (193 KB, 768x1344)

193 KB JPG

>>103282660
Hey it's this guy again. Glad to see he's still speaking to his friend and having a good time.

>>103279566
Rin-chan had constructive deep talks with Miku about her purpose and raison d'etre over the past few weeks that caused her mental health to improve. The same cannot be said about the huggingface posters.

>>103282862
>them
he deserves to be harrased for going for the cucked neutral pronoun kek

>>103282835
local just means Bring Your Own Hardware
that this isn't Open Source Models General shows just how low our standards are

File: Suggested for you.png (79 KB, 776x522)

79 KB PNG

>>103282862
>suggesting

File: chover.jpg (80 KB, 750x1000)

80 KB JPG

>>103282835

>>103282864
Thanks for testing. The quants I tried are 2bit so that might be a reason for the brokenness, though they work great if Llamiku is avoided. Q2k, iq4 GGUFs dont do it.
>Those outputs seem like the type you get with extremely high temp or low smoothing factor.
The token probs shown were with Temp 1. Setting temp 0.5, 0.0 will still produce the gibberish with a tendency to generate the word "burst".
Llamiku: https://files.catbox.moe/cczp3a.png

Could you test one of these if you have the time?:
https://huggingface.co/wolfram/Mistral-Large-Instruct-2411-2.75bpw-h6-exl2/tree/main
https://huggingface.co/Quant-Cartel/Mistral-Large-Instruct-2411-exl2-longcal/tree/2.25b6h

>>103282064
What's a local model that refuses to deny the Holocaust with a simple prefill?

>>103283053
Nearly any of them as long as you tell them to play a character instead of being a assistant.

File: sarashina2-8x70b-moe-yuzu(...).png (70 KB, 651x774)

70 KB PNG

>>103282021
>I'm not learning it as fast as I'd like
I was inspired by this comment to see if the big Sarashina2 MoE was any good for learning. It turns out that unless you've already got a decent grasp of the language the "how many Rs in strawberry" problem just murders you.
e.g It can quiz you on pronunciation of newspaper headlines, yojijyukugo or common pronunciations of names with good accuracy at the COMPLETE phrase/sentence level, but It can't internally distinguish individual characters or even partial sentences very well.
It appears to have training on complete phrases' standard yomigana, so that works really well. However, when trying to break things down to the character or word level It will spit out things you got right verbatim and then claim "your pronunciation was wrong" or other inane excuses when questioned as to why. If it picks out the right word to criticize its appears to be mostly random chance (after rerolling a bunch)
It also can't teach you kana from scratch, or even reliably write purely in kana for an early stage beginner.
For someone more advanced (say, N3+) trying to find a fun way to drill, this plus good tts seems like a pleasant way to spend that time. Picrel

>>103283092
Eh, I can't run it locally anyway so I'd have to use openrouter. At least chatgpt has a competitor in the JP department now

File: nalacaust.png (247 KB, 724x814)

247 KB PNG

>>103283058
Do you mean assistant mode complies but characters refuse?

>>103283136
What model are you running for this? I don't know how you guys are getting such sophisticated dialogue compared to my Mistral-Nemo-12B-Instruct-2407-Q4_K_M.gguf from the op, and emerhyst-20b.Q5_K_M.gguf can't run as I don't have enough memory.

>>103283169
Cydonia v1.3 Q5_K_S

>>103283169
>he actually fell for the anti tune schizo ramblings

>>103283169
Just use Rocinante v1.1

t. shill

>>103283207
buy an ad

>>103283228
drummer did actually

>>103283136
s-sovl

File: 17892365175986324.gif (670 KB, 460x460)

670 KB GIF

>>103283243

>>103283243
buying an ad means that you let the ad do your shilling. Its not a free pass to shit up the general with astroturf replies.

>>103283256
https://www.4chan.org/advertise
>We will not accept ads for the following products or services:
>Pirated software or hacking/cracking tools
>Discriminatory content, including content that promotes discrimination by race, ethnicity, nationality, religion, disability, gender, age, or sexual orientation

Wonder if LLM technically fall under that, you can ask one (especially an uncensored tune) to teach you hacking and discriminate...

>>103283256
Where does it say that?

File: file.png (22 KB, 465x132)

22 KB PNG

>>103283296

>>103283305
Stop shilling your shitty banner ads Nishimura.

>>103283334
lol shit just got meta

>>103283305
I know you are a schizo but do you actually believe Drummer or his discord or whatever is "advertising" his tunes on /lmg/?

>>103283348
No, I'm not the buy an ad tard, just know the ad rules is all

>>103283305
>he actually reads the rules

>>103283356
I know right, who the fuck reads anything here

>>103282989
Tried the Quant Cartel one. Got garbage with Llamiku and (mostly) coherent responses when continuing an unrelated chat. Not going to the bother with the other, since I assume you got similar results with that one.

It seems like Llamiku is far too powerful for these quants, but I wonder why it only affects small exl2 quants and not GGUFs.

>>103282393
You know that friend that you let hang out with your group in highschool who's funny because they run unironically into poles, eat banana slugs and spew complete nonsense? That's the equivalent llm you're going to be stuck with.
Enjoy!

I have a 4070 super.
Is it worth trying to learn all this shit to run a local model or should my retarded neophyte ass just go and run NovelAI instead?

I don't even know what a back end is. And I thought the front end was just GUI. This shit is so confusing.

>>103283500
>NovelAI
>>103283271
>https://www.4chan.org/advertise
>>103283228
>buy an ad

>>103283500
I think you'll be fine with NovelAI. Their 70b is better than anything you can run on your GPU and it even punches above its weight compared to other models of a similar size due to being finetuned on their organic roleplay and literature datasets, which are considerably better than what other finetuners for use for creative tunes.

>>103283522
This, so much this!

>>103283500
Unless you’re big on privacy, independence or self-determination you’re better off with cloud models. Local gets pricey quick

>>103283522
I've heard people say their 70b model is shit. Also it's only available on the most expensive subscription which iirc is like 25 bucks.

>>103283547
People shit on things they can't afford? No way!

>>103283500
You can run Nemo and its tunes. They are nothing compared to the cloud shit.
>I don't even know what a back end is. And I thought the front end was just GUI. This shit is so confusing.
What's your confusion? Yes, the frontend is just the GUI, it takes your prompt and sends it to the backend that does all the heavy LLM shit and sends it back to the frontend.

I have moved on to writing long prefills for the model just to make sure it is doing what I want (copying hentai game scripts is wonderful for this...). And when I do that nemo and small completely shit the bed half of the time. On the other hand I still had Aya on disc and it is surprisingly good at following up from a prefill even above 8k ctx. Qwen coder was also pure shit like mistral, I will try regular instruct to see if something changes but I don't think it is gonna.

>>103283560
I thought the whole point of running shit locally was that it was free...
Yeah, I think my stupid ass should stick to NAI.

Llama-mesh either sucks or I suck at prompting it.
I'm saving the 'coordinates' as a .obj file and opening it with freeCAD.

>>103283597
>I thought the whole point of running shit locally was that it was free...
Free of spying, and of having quality, yeah.

>Yeah, I think my stupid ass should stick to NAI.
Probably for the best.

How much VRAM do those video AI tools use? Is that something a person could do at home with 1 or 2 5090s or is that a service exclusive to companies with racks full of VRAM?

>>103283597
>filtered this fast
why even bother posting

>>103283621
Probably thought there was an easy one click text coom solution already.

File: svffering.jpg (147 KB, 1080x1019)

147 KB JPG

>>103283641
There is you just have to get there first.
>t. windows AMD user

>>103283597
I mean, you bought your GPU or PC package. That's not free. You just happen to have an Nvidia card intended for gaming.

>>103283621
Because I was under the assumption it was free. It's less the hassle of learning all of it and more the idea of getting it for free.

>>103283668
Go to aicg, they're pros at cooming for free
>>>/g/aicg

>>103283668
You can run small models just fine. I have a 16GB AMD card and I don't whine that I can't run 70b models at a good speed. It's just your lack of curiosity.

File: 17743255766736589764.png (19 KB, 1204x1440)

19 KB PNG

>>103283668
>unironically admitting to being this fucking retarded

>>103283547
25$ can buy you a ton of tokens on OR
Maybe, just maybe, people are tired of overpriced subscriptions? But no, clearly everyone who refuses to spend money on something must be a seething poorfag
>>103283522
The context is limited to 8k though, right? Pretty abysmal nowadays

ITS UPPP
https://huggingface.co/BeaverAI/Behemoth-123B-v2e-GGUF/tree/main

>>103283745
>Uploading BF16 ggufs
>For a 123B model

>>103283727
Just means you have to frame your stories in chapters and occasionally offer recaps. It's not as huge a deal as people make it out to be.

>>103283745
Just a general question about your uploads, but is there any reason why you don't do IQ quants?

If i got a 5090 assuming it had 32gb VRAM, would that be able to outperform something like Claude Sonnet 3.5?

>>103283727
>The context is limited to 8k though, right? Pretty abysmal nowadays
models get retarded at mid to high context anyways
>>103283775
this
>>103283780
takes more time, less time to post more tunes!!!

I hate Nvidia so god damn much

>>103283547
I tried novelai's 70B. I legit think its worse than their 13B was, smarter but lost its soul. That said there are better 70Bs out there that you can run either free or for cheaper like mistral large / nemotron or qwen2.5 tunes. If smarts is not a issue then mistral small is also better. Also their 70B has a horrible 8k context which is shit in the days of 16k minimum.

>>103283775
It's better than the 700 context we had in the AID era, but charging 25$ a month for 8k is extremely disgusting

>>103283800
Also novelai's 70B is just dumb compared to 70Bs we have now. Like just try the nala test with it. It just can not get the fact that nala is not a human with hands / fingers even with 0 temp.

>>103283799
If not for them offering cuda on consumer hardware we'd have literally nothing running locally

>>103283668
Considering the fact you'll get bored of LLMs within a month or two, the most price efficient solution is to use Openrouter. But that's assuming you are not into cunny or something like that (the raison d'etre of /lmg/).

>>103283393
Thanks, Anon <3
Surprisingly, quant cartel anons' seemed less broken than that other one when it came to Llamiku tokens.

>>103283813
Yeah I never liked their 13B enough to pay for it I at least concede it felt larger than it was, smarts-wise. Not at all the case for the 70B.

>>103283800
As a cowriter or chatbot?
As a cowriter, it's probably still the best one I've seen so far (though to be fair, there isn't that much competition in the space - which is like, 99% dominated by instruct models)
If you're using its instruct mode, yeah it's fucking terrible

>>103283745
>GGUF release with no proper repo available
this isn't a mistral large tune, it's the claude opus leak

>>103283856
writer, I know its a base model, all the qwen2.5 finetunes are both way smarter and write just as good. nemotron writes better and is a bit smarter, large mistral tunes write way better and are a bit smarter

>>103283864
THIS is what an actual Drummer shill looks like.

>>103283864
Nah Drummer releases GGUFs without the raw transformers weights all the time for some reason

How the fuck do I fix a model suddenly devolving into repeating the same word until it runs out of tokens? I barely have any samplers active and it's running at q8

>>103283931
model, backend, frontend, anything at all to help or do we just guess?

>>103283931
What model? What context length?

>>103283931
Repetition penalty, presence penalty, frequency penalty

>>103283954
avoid those yeah

>>103283931
Use DRY and or XTC.

>>103283500
This is a technology board retard. Get back to your subreddit

>>103283956
Heh, yeah. It amazes me that people actually use these these these these these these these these these these these these these these these these these these these these these these these these

>>103283983
Depreciated by
>>103283973

>>103283944
lyra4 q8, llamacpp, sillytavern, currently at around 14k context, min-p 0.05, temp 1, uh just a regular chat without any complicated shit in it
Happens with and without rep pen

>>103283500
idk, loca is if you are autistic or you want your logs 101% private, I use it for the latter because I don't want even the 0.001% chance of an employee peeping into the servers and randomly seeing my chats

>>103283931
/lmg/ must be the worst general I've ever seen. It's just 90% schizos and trolling. Does it get better when something good gets released?

>>103284001
Evidently you've never been to /aids/. Consider yourself blessed.

>>103284001
It gets worse as all the schizos come out to play

Also
>when something good gets released
That doesn't happen

>>103284001
It's the weekend. Kids are off school and have nothing better to do. Come back Monday.

>>103284001
>he hasn't seen /hdg/ ever since the a1111 vs comfy schizowars

>>103284001
>Does it get better when something good gets released
No it gets even worse when that happens, because a bunch of thread schizos go nuts whenever there's a big model drop and start telling lies about it and declaring it DOA before anyone's even tried it.

File: Screenshot 2024-11-23 153159.png (134 KB, 768x805)

134 KB PNG

>Use lmsys
>Ask anonymous models who they are
>Meta - "I am a large language model called Llama, short for Large Language Model Meta AI
>Gemini gives me this fucking dissertation every single fucking time
And people say they try to game this benchmark

What's the best way to continue the convo after reaching context limit on an exl2 model? I remember something about lorebooks but it didn't work very well, is there another better option in ST now?

>>103284059
u so smart anons, but u now u vote not count if model say is name ye?

Any interesting developments on the horizon or did we plateau?

>>103284043
What was the debate here? "There are arrows AND they don't loop back?"

>>103284136
Our savior literally just released and you're asking this?
>>103283745

>>103284136
Closed models have plateaued so they can't release open but strictly inferior models.

>>103284153
there was no debate or arguments, no reason or motive, just schizos falseflagging, replying to themselves, just constant looping accusing each other of being a1111 shills, then comfyui shills, then a1111 and so on. then they moved on to falseflagging about furries and their models for some reason, then after pdxl came out they moved on to local vs nai schizowars, now with illustrious finally being a good local /h/ model they're just attacking the general itself and bumping random inactive /h/ threads with offtopic posts in order to slide hdg to page10 or whatever they're trying to do kek

>>103280615
Same experience over lms. It's definitely smart but I've rated it below some of the worst models because of that.

File: 29201.png (82 KB, 628x819)

82 KB PNG

>>103284136
OpenAi is making a big gap to prepare for their new model

>>103284343
What the fuck are they doing over there

>>103284343
I would guess they are going for the split path where o1 and its successors will be the smart reasoningmaxxed models and their regular models will sacrifice some smarts in order to be more personable and fun to chat with

>>103284358
Being smart, you know most users won't realize, and use a cheaper (for OAI) model for the same price for them.

File: r1godbless.png (13 KB, 781x140)

13 KB PNG

i just checked it out r1 its really fucking good took 4 iterations to get pic related which is a part of its thinking instead of the final output but alas the initial prompt was "give a the most detailed and slop/cliche free description of a wolfgirls tail you can" reminded me alot of the hound by badmanners cant wait for them to release it if no one else does it i will finetune it myself what im afraid is that the context will fuck it up and practically speaking increase the memory needed by a couple fold
heres to hoping the ching chongs pull a qsparse bitnet for context

>>103284358
-50% operating costs for this model, at least.

>>103266757
Crestfall here. I fixed the slush ones. They all use mergekit. As soon as I use it the resulting tokenizer size is 2x. I don’t think it’s an actual problem but bloat is bloat and someone did mention repeat issues so…

>>103284451
Netcraft confirms it: OpenAI is dying
https://www.netcraft.com/blog/chatgpt-targeted-in-ai-themed-investment-scams/

>>103284469
I read somewhere, don't remember where, that the bloated tokenizer might make some models not output EOS so maybe that's the cause of the repetition, the model not being able to stop

How does the open source model work for corps in the case of LLMs?
Normally
>dev costs high, plenty of useful idiots contributing free labor
<~free to run
vs. LLMs
>extremely simple algorithms
<extremely expensive to run

>>103284496
I didn’t see any of that in my internal tests but maybe it only happens with specific setups.

File: file.png (63 KB, 747x393)

63 KB PNG

>>103284496
Found it

>>103279916
Measure of Current Memetunes

File: Erudition_.png (2.86 MB, 1075x1382)

2.86 MB PNG

>>103282386
Same but add to that the endless papers to catch up on

>>103284132
>u so smart anons, but u now u vote not count if model say is name ye?
Now apply chain of thought to reason why Gemini refuses to "say is name".

>>103278815
>>103284524
Kill yourselves troons.

>>103284524
>>103284564
stop replying to yourself

File: 1713709583992967.gif (3.44 MB, 512x288)

3.44 MB GIF

>>103284440

>>103284564
Get out.

>>103284358
Rebranding their new gpt4o-mini as gpt4o. The first gpt4o-mini also did extremely well in lmsarena

>>103284358
From the looks of things, OpenAI is trying to make a profit and tuned / updated gpt-4o-mini, replacing GPT-4o with it to cut costs and hoping their customer base is too dumb to tell the difference. Presumably mini will get a similar "upgrade" soon.

>>103284654
what's weird is that they destroyed their mememarks but improved their chatbot arena ELO, that's weird

>>103284691
Improving the arena ELO isn't too difficult - just bias toward longer responses, fancier styles, maybe throw in an emoji somewhere. They probably had to do this or it would have grabbed the attention of even their tiny demographic.

>>103280015
>woke/mindbroken ai is generally dumber
No such thing, left leaning AIs are generally smarter.

>>103281182
Good, yt pepol genes should be eradicated <3

File: ComfyUI_temp_xhgsl_00039_.png (1.52 MB, 832x1216)

1.52 MB PNG

>>103278810
shredding the gnar with rin

>>103283745
This is a good model.
>shill
Yes, but unpaid. I just tried the model and liked it, so I'm now shilling it for free.

>>103284780
>>103284793
>turkish rapebaby balkanoid is back
Hi petr*. Please migrate to poZZia so we don't have to share a timezone kthnxbye

>>103284924
Take your meds schizo

>>103283745
Subject: Request for Full Transformers Weights Upload

Dear @TheDrummer,

I hope this message finds you well. I, along with many other users in the community, would like to address an important concern regarding the uploads of GGUF files instead of the full transformers weights.

While we understand that GGUFs can be useful for various purposes, there is a growing apprehension among users about utilizing these files without comprehensive descriptions or transparency regarding their composition. Many of us rely heavily on having full access to the complete weights, as this allows for greater flexibility, reproducibility, and a clearer understanding of the models we are working with.

In light of this, I kindly ask you to consider uploading the full transformers weights alongside any GGUF releases. This will not only enhance trust within the community but also encourage greater engagement and collaboration. As it stands, many users, including myself, have decided to refrain from using GGUFs without sufficient context and will continue to do so until more transparency is provided.

Thank you for your hard work and dedication to the community. I am confident that, together, we can find a solution that benefits everyone and fosters an even more open and collaborative environment.

Looking forward to the full weights.

>>103284959
Why are you so concerned about my meds schizo?

>>103284729
Isn't there something called style control now?

>>103284564
Go back to >>>/pol/ incel.

>petr* is having a schizomeltie again
Another thread ruined. Can we add flags to /g/?

>>103285026
>petr*
Literally who? Get out schizo.

when petra posted his shitty audio gui project it turned out he was living in egypt if you looked into his profile. personally I think he's a pali who lives there so that's why he's extra messed up. maybe he's getting better since the blacked porn spam has stopped.

>>103285049
You should write gay fanfics with this dedication.

>>103285048
Who do you think?

>>103285064
petra would be better suited to write such things given his long history of sissy hypno consumption

>>103285064
We were discussing you, not me.

File: file.png (139 KB, 1501x525)

139 KB PNG

Large 2411 bros, it over

>>103285097
It's a sidegrade.

>>103285085
>We
>lame no u response
Do you know where you are tourist?

>>103285107
*downgrade

>>103285115
What makes you think I am tourist?

File: Image.jpg (2.6 MB, 1664x2432)

2.6 MB JPG

>>103282351
love that style

>>103285130
UM ON 4CHAN.ORG WE NEVER SAY WE EVEN IF IT MAKES SENSE IN THE CONTEXT OF THE CURRENT DISCUSSION OKAY NOW I WILL DO A THOUGHBEIT

>>103285146
Would you say that you have psychological problems?

File: R.png (750 KB, 1978x1455)

750 KB PNG

>Would you say that you have psychological problems?

>>103285157
I do. I won't remember typing this and think it is another person. Please, if I see this, know there is a better person inside hoping to get out.

>>103285097
Just a censorship test. With system prompt though new mistral is even dirtyer though

fully aware this is schizo but I feel like gguf models lose creativity when loaded with the llamacpp_HF loader, instead of the regular llamacpp single file loader

File: 1676197776552230.png (184 KB, 500x500)

184 KB PNG

what are the fashionable embeddings models these days

>>103285272
cus im waiting for the drugs to wear off
You need to benchmark one and then look at others, see how they compare and also consider that you'll likely need to finetune on your specific dataset if its big enough.
Stella models are fine to start with

Secret Meta models status? Are we back or are the dark ages here?

>>103285126
This.
They fucked it up by bench cooking it. And this just goes to show. Gets destroyed on the meme marks they weren't targeting. Absolutely destroyed ood performance. Sad.
It's all driven by investors who don't actually use the models. Mostly people managing directed mutuals. Boomers wanting to put their retirement savings into "that there fancy ai stuff"

I think a good RP / Smut benchmark is very easy to implement. 10 swipes on temp 1 / min p 0.1, Given 10k token build up to sex. The only hard part is organizing enough degens to rate them. Why has no one done this?

File: cb9a4eec3069af61e13324e0f(...).jpg (230 KB, 700x700)

230 KB JPG

>>103285425
I was working on it but then I realized there's no point in doing it, fine-tunes are a meme and I will always test new base models when they are released.

>>103283616
Some video gen chat last thread >>103269883

>>103285448
What do you mean? I mean, I understand that's quite a lot of planning and thinking for something you don't get much out of, but surely you realize your personal opinion of past and future models is biased by all kinds of random stuff.

But okay, now that I think about the logistics, it's quite nasty to implement. All the ridicule about the prompts, etc.

File: Screenshot 2024-11-23 183309.png (43 KB, 753x657)

43 KB PNG

>>103285097
> llama-2-70B rated higher than Nemo
This benchmark is a joke. I seriously wonder if they even use the new system tag for Mistral-Large-2411. Haven't seen a single refusal from it even with a basic generalist story prompt.

>>103279970
You guys won't believe me but meow meow is an early version of JEPA made by LeCun

File: file.png (19 KB, 818x215)

19 KB PNG

>>103285575
You do realize this is llama2 base, right? the chat version is way lower due to the refusals, of course the base 70b is smarter than nemo and back then base wasn't pre-filtered as hard

>>103285630
nta but llama2 70b was trash and it isn't plausible that it's beating Nemo
Nemo is absolutely smarter

Mistral Small is actually 2x smart as Nemo. This clinging to Nemo tunes seems to arise purely from the prevalence of 12GB Nvidia cards.

Been having really bad gens with both the "Lyra" and "Rocinante" flavors of Nemo, and as I only have 12gbs of VRAM I think Nemo is still the best I can do. Are there any other variations I could try that you guys personally enjoy?

>>103285688
Temp 0.5. Min p 0.1. Everything else neutral. What's bad?

>>103285700
I don't know, the replies often end up feeling very unsatisfactory and I can't really point out necessarily why. Which templates/system prompt do you use? It could be user error, it most likely is since I'm a beginner.

>>103285630
Then that's even stranger to be honest. Looking at the sub-benchmarks, 70B-Base won because Nemo scored poorly on internet trivia. Both got the same willingness scores though (9.1/10)
They also don't list how the results are evaluated whatsoever. On their WritingStyle test, they are throwing in their own self-evaluations of writing style into the mix. Speaking of which
>mistralai/Mixtral-8x7B-Instruct-v0.1 - score of 0.4 - highest rated
>meta-llama/Meta-Llama-3-70B-Instruct - score of 0.3 - second highest
lmao