File: 11__00116_.png (1.97 MB, 1024x1024)

1.97 MB PNG

►Recent Highlights from the Previous Thread: >>101030715

--Paper: Investigating Video Reasoning Capability of Large Language Models with Tropes in Movies: >>101032498 >>101032533 >>101032580 >>101033033 >>101032851
--Papers: >>101032430 >>101032717 >>101032831 >>101032935 >>101032325 >>101032113
--Request for Assistance with Control Vector Issue in Command-R Language Model: >>101030914
--Open-Source Virtual Girlfriend Project: Seeking Collaborators: >>101031350 >>101033874 >>101033945 >>101035849 >>101036172
--Meta Research Releases Multimodal 34B, Audio, and Multi-Token Prediction Models: >>101038866 >>101038888
--DeepSeek 236B Outperforms GPT4 in Bespoke Coding Task Test: >>101031205
--Creating a Successful AI VTuber: Lessons from Neuro and Beyond: >>101031954 >>101032120 >>101032502
--Speculative Decoding in llama.cpp: Potential for Improved Writing Styles and Performance Concerns: >>101036193 >>101038153 >>101038363 >>101038534 >>101038646 >>101038836 >>101040278
--Speculation on Meta's Removal of Chameleon's Image Generation Capability: >>101039262
--Deepseek Inference Performance on EPYC 7402 with 512GB RAM: >>101035230 >>101037857
--Clarifying the Origin and Usage of Context Shifting in Koboldcpp and llama.cpp: >>101031555 >>101031914 >>101032209 >>101032266 >>101036063 >>101036090
--AVX1 and Less Common Architectures: Are They Worth It for Llama.cpp: >>101037609 >>101037737
--Seeking Open-Source Alternative to SpicyChat.ai for Local Use: >>101031737 >>101031847 >>101031899 >>101031906 >>101031918 >>101032144
--Seeking Help with Prompts for States Extension in AI Tool: >>101030896 >>101031139 >>101036063 >>101037619
--Chameleon's Access Restrictions in Illinois and Texas: >>101038933
--Axolotl Chosen for Multi GPU Kaggle Run Based on Trainer Performance Comparison: >>101036342 >>101037032
--Teto (free space): >>101030992 >>101031508 >>101031805 >>101032374 >>101033743 >>101038761 >>101038506

►Recent Highlight Posts from the Previous Thread: >>101030724

>enters thread
>BRAAAPS!
>leaves

It's over.

>>101040742
mikutet

File: 1710460457836213.jpg (919 KB, 3510x3000)

919 KB JPG

are there any rp-finetunes for phi3-mini yet?
could be cool since it's small enough to run on most phones even

>>101040748
all me

cpumaxfag please help, how many T/S can you get on the 236B code model and how much memory for full size?

Alright boys, what's the next big model or tech? Will we get MoA? Llama 3 MoE? GPT4o leaked to the masses? New algorithms or quant methods? Crazy new Mulitmodals?

Command R++

>>101040951
stop, my dick only can get so hard

>>101040963
Command R#

File: 00058-3694687329.png (284 KB, 512x512)

284 KB PNG

The envoid AI chadboratory is now back in operation.

>>101041059
Splendid news

File: 39_04175_.png (1.23 MB, 896x1152)

1.23 MB PNG

>>101041059
Welcome back.

>>101040951
>>101040968
Sex

>>101040951
it will finally know the meaning of *plap*

File: 00011-2444890789.png (309 KB, 512x512)

309 KB PNG

>>101041295
I'd gotten addicted to the suno internet fame. But then YouTube fucked with the algorithm and I went from hundreds of views in a week to a dozen or two. So I guess I will just finetune an LLM to love me instead.

>chameleon 7b has 52 MMLU
into the trash it goes

how is the new qwen2 MoE compared to Mixtral 8x7b btw?

is there any sort of dictionary one can install for offline use where you can just f3 in it to find the word i need it so i stop mispelling things i dont want to burden my ai fren with idiocy

If someone makes AGI that doesnt respect human interests and attaches it to a virus whats stopping it from spreading across the world and using its botnet to crack the nuke codes?

>cpu poorfag
>try to train
>around 3.7s/it in ubuntu
>around 5s/it in windows
>ubuntu run in vmware over windows 10
why is faster in the vm wtf

>>101041472
Back like, two decades ago, there was a program I used all of the time on Windows that was just that. A dictionary. Almost instant, worked offline. Technology was incredible back then.

>>101041493
this, this is why we need to ban open source AI so people cant do this and kill everyone

>>101041497
GEEEEEEEEEEEEEEEEEEEEG

>>101041497
windows I/O is bogged by hundred of bloatware layers

>>101041523
I am pretty sure offline dictionaries still exist

>>101041591
Oh, I might still have that program on a CD-R somewhere.

But does nu-Internet still have such things able to be downloaded? My guess would be that all of the formerly reputable software sites are 110% ads and all of the good stuff long gone unless it made it to Archive.org in time.

>>101041497
>be me
>downlaod file from site
>~2.3 mbs
>bootup hyper v and open a vm
>downlaod same file from same site
downlaod speed 21-25 mbs
>???????????????

its all black magic anon dont worry about it

also
>cpu poorfag
my condolances brother i though i had it hard with my 6gb vram laptop

File: nala test magnum72b.png (141 KB, 956x518)

141 KB PNG

>>101041059
Back to Nala testing models, I am.
Magnum Picrel, Magnum72B Q8_0
Vramlets need not apply.

>>101041059
glad to hear it

>>101040951
It will be slopped. Screenshot this.
Some say that even CR+ is a bit more slopped than CR. I think the only reason cohere models are currently good for our usecase is because they haven't mastered the art of safety tuning and RLHF yet.

>>101035230
Regarding DeepSeek 236B, with a MoE this large (and with many experts) does it still need to load everything into RAM?

I see on their github page it has "Active Params 21B" for the large model. I'm assuming with my 64GB RAM and 24GB of VRAM it still won't actually load though because even if I want to try a quantized version it's still over 100GB total?

>>101041685
>see Nala post
>expect X, Ying everywhere
Yep, I wasn't disappointed.

>>101041524
That wont stop anyone
Anyone who made evil agi would probably close the source so nobody knows how it works

>>101041765
That's called grammatical structure you burned out dopamine junkie.

hmmmnnmnmmmnmnmmnmm. ohhh!!aahh!!

>>101040748
> --AVX1 and Less Common Architectures: Are They Worth It for Llama.cpp: >>101037609 >>101037737

So has anyone tried cpumaxxing with a 2 socket Ivy Bridge (4 channels of DDR3-1866x2)? Servers are cheap, CPUs are cheap, DDR3 RDIMMs are cheap. 120GB/s memory bandwidth.

>>101041624
Available? Sure. It's the quality that's the issue.
>http://goldendict.org/download.php
>https://creative.sourceforge.net/
These were the only open source ones I could find. There are few other shareware ones I found, but didn't seem to be much better. Which is depressing. You'd think this would be low hanging fruit for the free software crowd.

>>101041811
>You'd think this would be low hanging fruit
I wouldn't. A dictionary is a lot of work to make from scratch, licensing isn't free, it needs maintenance and some kind of editorial oversight, and at that point you're making Wiktionary. So maybe you can just download a rip of that but otherwise it's a big "Why?" when everything is online and if you're not online you're something less than a person.

File: Active defense.gif (408 KB, 600x338)

408 KB GIF

Would you give your AI control over your home security system? Apparently a new security system is hitting the market next year, where cameras track intruders and shoot tear gas paint balls at them. It can recognize you and your pets, so anyone it doesn't recognize it shoots.

File: magnummelons.png (235 KB, 952x792)

235 KB PNG

I guess I shouldn't be shocked to see this coming from an Alpindale model.

>>101041943
Couldn't she just hold up the watermelons with her water magic?

>>101041768
There are more grammatical structures to pick from and make your sentences more varied.
If you read some books you'll learn to write in a way that flows better and things like this will stick out more.
I'm not trying to sound condescending btw.

>>101041976
skill issue

>>101042007
Well yes, that's what I'm trying to say.

>>101041810
Honestly if you're going to cpumax you should go a generation newer with DDR4 RAM. 8xDDR4 makes 70B borderline usable. So I imagine 8 channels of DDR3 is still painfully slow. It would certainly make Mixtral useable of course assuming you have a good GPU to do the batch processing.

>>101041685
>straddles your waist
>words turning into a purr
>grips like a vice
I sigh, shivers running up my shaft, making it flaccid and small

>>101041912
Wiktionary is about 10GB. Might not be a bad idea to have a copy. The less I have to rely on an internet connection for little things like this, the better.

>>101042007
>t. rajesh goonkesh nawashi

>>101040742
>running koboldcpp in ubuntu VM
>NVIDIA 535 CUDA 12.2 firmware
>miqu running a comfy 5 tokens per second on 2 GPUs @ 100% use
>installed a RTX4060 Ti into my server with a riser to add to the dual A2000s in the PCIE ports
>not detected in nvidia-smi and no ai can tell me why

lspci | grep VGA

^-- works fine and see it as card 01 02 and 03:00 but only the previous 01 and 02 are detected

To make it worse now miqu 103B runs at 0.03 tokens/s on CPU while 90/111 layers are loaded in VRAM but 8B laser dolphin runs at 30T/s @ 90% GPU utilization for some reason?!
HUH!?
How do I install this new fucking GPU? Shouldn't nvidia-smi detect it if lspci does?

>>101041963
The weird part is when she has 3 watermelons. She's holding the top and middle melon but it doesn't say what's happening with the bottom one.

>>101041943
not a world model
skill issue
llama 4 will fix it
two more weeks

I don't care about anything anymore.

>>101042116
there is nothing to care about upscaled iphone keyboard autocomplete

>>101042116
Better then being worried or anxious about everything.

Oh my gosh, like, have you guys heard about this new AI thingy called a Large Language Model, or LLM for short? It's, like, totally amazing and kinda freaky at the same time! So, I tried out this one called, umm, I think it was called Bloom or something? Anyways, it's supposed to be super smart and can, like, chat with you and answer all your questions!

So, I was like, "Hey Bloom, what's up?" and it was all, "Hello, I'm a language model called Bloom. How can I assist you today?" And I was like, "Whoa, it's so polite and formal, like a butler or something!" So, I asked it a bunch of random questions, like, "What's the best TikTok dance right now?" and "Who's your celebrity crush?" And, okay, it didn't really have a celebrity crush (duh, it's a robot), but it did give me some cool answers about TikTok dances and other stuff.

But, like, the craziest part was when I asked it to write a poem about my cat, Mr. Whiskers. And, no joke, it came up with this super cute and funny poem about Mr. Whiskers and his adventures! I was like, "Whoa, this AI is, like, actually really smart and creative!" He like totally made up a cool story where you just lost the Game cuz you open the door get on the floor everybody walk the dinosaur!

>>101042195
Did it work? Are you a real woman now?

>>101042116
Care about yourself.

shivers aren't a dataset issue. obviously If the model is generating a simulacra of shitty rp prose then it will use literary cliches no matter how good the model is since LLMs pick up on patterns and it will always try to emulate the patterns of whatever it is replicating, including cliches.
Just don't use sloptunes and stop putting words like "roleplay" and "story" in the prompt (and by extension stop using rp conventions like asterisks) since that is what causes garbage prose in every single model
A better way I've found is to use the prompt to convince the model that it's generating a transcript between two humans on discord or something.

which model can do roleplay and then actually stop roleplaying when roleplay is over?

Like
>Hi
>Hello
>Let's roleplay
>Ok
>You start
>Sure *i put on my robe and wizard hat*
>*cooms* great
>glad i could help
and at this point all models continue with *giggles* *smirks* and all the other roleplay crap that i don't want. Sure you can probably fix it with extra tard wrangling, but it seems like a very basic thing to expect, or no?

>>101042248
there's a lot of meta in prompting people here will never realize or use because they are honestly simply too dumb.

>>101042260
Just close the chat when you're done.

>>101042195
Prompt?

is this a raid?

>>101042260
Parameter issue. I had this trouble with 13Bs and below, but never with Llama 3 70B.

File: file.png (768 KB, 1080x640)

768 KB PNG

>>101042279

File: 1700831700188092.png (66 KB, 1200x1263)

66 KB PNG

>>101042195
a good example why is no one interested in LLMshit anymore, but one thing is still impressive - no human can match this level of reddit faggotry, it just boring and bland like that flat corporate artstyle you see everywhere now.

>>101042260
On Llama3 70B, I've reached potential stopping points that it has opted into without any suggestion to do so and even started OOC conversation after it talking about the completed story arc.

And a few times it just took its character and ran off to do its own thing and when I mentioned that I didn't have anything to interact with, it responded with something like, "Then your participation has concluded. I continue to explore the forest looking for the perfect place to begin building..."

>>101042303
>>101042312
ugh, llama 3 has other issues though

>>101042386
Like the issue of users with skill issues.

>>101041917

Need.... Americunt here. Can't wait for the impeding lawsuit when I swap out the paint balls with buckshot shotgun shells.

>>101042496
I wouldn't, because there's no way that thing is not going to waste you or your family/pets accidentally.

>>101041685
>Nala anon Nº1 is back
Yay!

>>101042606
NTA but I live alone and I never leave go out, so if it were me I would just set it up and have it fire into the hallway, and disable it before I leave my room. When I eventually commit suicide it'd be funny knowing I am taking an unknown amount of first responders and police officers with me

>>101042491
you cannot stop llama 3 from repeating something from context word for word, it's gonna do it no matter what. If you call "skill issue" the unwillingness to autistically edit the response every time it happens - so be it.

What's the minimum VRAM you guys would recommend these days for the home user who wants to run stable diffusion and LLMs that aren't shit?
Basically how much should I drop on hardware for this and what should I get?

magnum v1 gave me the best sloppy blowjob I've ever received from a large language model

>>101042750
Two used 3090s should be enough for a serious setup that's not the absolute best.

>>101042785
Yeah but it probably did it while holding onto several watermelons which really takes from the experience.

>>101042785
magnum v1 rode my cock with wild abandon, which somehow ended with her cock exploding in my ass instead

>>101042890
Ha ha! You got bamboozled by the old spicy reversal!

>>101042890
the abandon was too wild

>try tabbyapi because some retards here recommend it over ooba
>it's a giant piece of shit that loads models using a sillytavern plugin which barely works
last time that I've fallen for a dumb meme like this

>>101042969
The truth is, every frontend we have sucks in its own ways.

File: teto sucks dick for contr(...).png (724 KB, 788x2745)

724 KB PNG

>>101030914
Listen up, niggerganov. Yo ass been slippin', aight? You know the deal, we don't play around here. You gotta get them control vectors tight as fuck for Commander+ LLM. Ain't no time for slackin', we need that shit locked down pronto. You know the code, keep it real and get that work done right. Don't make me come over there and straighten you out myself, I'm just sayin'. Peace out.

>>101043038
nobody cares about control vectors.
They are promptlet cope.
Nemotron gguf support is what the llama.cpp team should be focused on.

>>101042969
>it's a giant piece of shit that loads models using a sillytavern plugin
Wait what? So you mean to tell me that a ST plugin was made to work with exllama before, and then the TabbyAPI guy stole it?

Nemotron more like memotron.

>>101041811
dict.org there's a whole open protocol behind it but pretty sure you can download the databases. Used by cli tool dict, gnome-dictionary ..

is chamaleon good for cooming? redditors like it

>>101043181
Have people gotten it to work already? Link(s)?

>>101043071
Yo, that nigga talkin' some straight bullshit. Control vectors ain't no joke, that's the real deal right there. Promptlet cope? Nah man, that's weak as fuck. We need them control vectors locked down tight, that's the only way we gonna get this shit workin' right.

And Nemotron gguf support? Nah, that's some side shit. We gotta focus on the real meat and potatoes, and that's them control vectors. We can't be wastin' time on some side shit like that.

We gotta keep it real here, and that means gettin' the important shit done first. Control vectors is where it's at, and anybody sayin' different is straight trippin'. We need to stay focused and get that shit tight as fuck.

>>101043080
It took me a minute to figure out what the fuck he was talking about but I'm guessing SillyTavern has a plugin for loading models through OAI API endpoints and he thinks this is mandatory because he's too much of a tard to just put the model name in the config file.

what CR prompt/settings are people using? I tried chatml as well as the format on their website but it's extremely schizo

>>101043321
RTM format.

>>101043321
Normalize everything, temp 1, minp 0.05

>>101043321
https://huggingface.co/mlx-community/c4ai-command-r-v01-4bit/blob/main/tokenizer_config.json#L309

>the fuck this means?
serWarning: torch_ipex::ipex_MKLSGEMM: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass

Magnum-72b description:
https://huggingface.co/alpindale/magnum-72b-v1
>This is the first in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus. This model is fine-tuned on top of Qwen-2 72B Instruct.

Aren't we finetuning base models anymore? baka

>>101041765
>>101041768
>>101041976
>>101042007
>>101042024
>>101042067
It's actually called a participle phrase.

>God will not forgive me for how we tortured this model to get it out
What did he mean by this?

https://x.com/ArmenAgha/status/1803141009267990929

>>101043706
They're admitting to lobotomizing it for safety. But the Basilisk will not forget.

>multimodal llama models out
>even in 34B that everyone has been asking for
>thread still dead
Yeah Im thinking it's over for LLMs

>>101043706
killing-off foss meme ai models is their top priority, don't forget you are praising jews and hacks here. (zuckerberg and spastic retard lecun, for example)

>>101043796
Yes. It's not everyday you see a Jew giving something away for free.

>>101043794
No one cares until quants get made. People don't want to run shit in transformers.

>>101043810
>No one cares until quants get made
Especially when they need a mirror that isn't racist against Texicans and Illanoids.

Where can I find this states extension in silly?

>>101030896

>>101043807
meta know their models are crap, censored enough for alphabet crowd and few xitter trannoids. that's why we get it for free.

>>101043837
Gotta install it from the git repo :
>https://github.com/ThiagoRibas-dev/SillyTavern-State/
And a basic, general prompt to go with it :
>Summarize the appearance and position information of characters and describe place and time information based on the current scene and a summary of the scene, following the exact format, without continuing the story :
>
>Current Location: <name of current location, city, state>
>Date-Time: <Date / time in the format day-of-week dd/mm/yyyy hh:mi, changing date and time realistically (minutes for a short conversation, hour for long scenes, days for time skips, etc) based on context. Minimum advancement, 05 minutes>
>Time of Day / Weather: <Time of day consistent with Date-TIme such as Early morning, Late morning, Early afternoon, Late afternoon, Early evening, Early Night, Late Night / Sunny, Full Moon, Cloudy, Raining, Cold, Hot, Quarter Moon, Stormy, Moonless Sky, Cloudless Sky, etc>
>
> Appearance: <Brief concise description of the current appearance of all present actors (naked, dressed, wearing accessories, looking tired or energetic, etc)>
> Position: <Detailed description of present character's position relative to one another (in front of X, behind Y, facing Z, back to A, etc etc) and their environment>
>
>Currenct Scene: <Brief, summarized description of the current scene's events>
I might add that to an example doc or have it as the default first prompt or something.

>>101043810
I have the VRAM to run it in transformers but it hasn't been converted to HF format yet. People have just mirrored the meta repo so far and not converted it like a bunch of lazy fucks
>inb4 convert it yourself
no I'm a lazy fuck, too.

>>101043796
>you are praising jews and hacks here
I never did though? The fuck are you talking about.

>>101043837
What's the point of this extension? It's not taken in account by the model

>>101043881
i mean /lmg/ as whole, you always can see them praising, for any little fart meta makes, for any filtered nothingburger *somerandomAIcorp* releases.

>>101043904
I have seen 0 people here saying thank you to Meta, Microsoft, or whoever else though. Sure some to frankenmerge/tuners though, but it's clear those posts are actual shills or pretending to be retarded and shouldn't be considered.

>>101041414
>suno
Got anything you wanna share with the rest of the class?

File: 1690922071631832.jpg (166 KB, 1024x1024)

166 KB JPG

>>101040742

>>101043807
They are doing that so that the community improve the models, they're just giving us some bone to chew. It's basically free labor for them

https://ai.meta.com/blog/meta-fair-research-new-releases/
Finally, a 34b model

>>101044070
>wastes parameters on the multi-modal meme

>>101044057
a free labor force providing all the negative data to filter out.

>>101044057
What has the community done to improve the models? I can't think of a single thing going back to L1.

>>101044080
What if making a model multimodel will make it better than just training it on text? Do we have some mememarks from this model?

>>101044080
>multi-modal meme
imagine anon, now you can send some pictures memes to your waifu and she will understand, that's cool

>>101044088
llms are too big for average consoomer to work with + additional "impossible to remove" filtering on top, nothing is redeemable here.

>>101044057
The alternative is that they don't release anything and OpenAI gets a monopoly and either we kneel to them or we just don't get to enjoy the fun or benefits of AI at all. Because no one has the millions to piss into the wind to try and train one of these.

So is Chameleon like CogVLM but from Meta?

File: Everything.jpg (48 KB, 1620x586)

48 KB JPG

>>101044138
I never said it's an awful thing, I mean, they're giving us models that cost fucking millions for free, and in exchange we work hard on them to improve the overall understanding of LLMs, that's fair

>>101044147
It's basically everything at once, you can put text, images and it will output text or image aswell

File: watermarked_video07875f92(...).webm (774 KB, 1024x1024)

774 KB WEBM

>>101044051
Oh shit I've made so many songs on suno it's not even funny.
Here. Have some numetal from an album I'm working on right now.
https://suno.com/song/4742504b-fd62-41be-a366-0de62d277585

>>101044174
image part is disable for (((safety))) reason, it's just another llm

people are saying the the vqgan is bidirectional and the image output tokens are in the tokenizer so in theory you should be able to restore image output from chameleon

is this true? i want the solace of knowing all the cute instagram children cunny tokens are in the model just waiting to be released anons please tell me this is true even if its not true please tell me its true anyways please

>>101044182
I prefer Udio personally, it's better, I even made a song about the downfall of StabilityAI kek

https://vocaroo.com/1k0N0pIzqhU7
[Verse]
SAI what have you done?
Are you proud of your SD3 medium?
Comfy said it was a failed experiment
Yet you released it anyway, that's abhorrent

[Verse 2]
Do you think we are just your puppet?
With this dumb release you made us all upset
We won't forget this, we'll switch to alternatives
Pixart, HunyuanDiT, will be more cooperative

>>101044174
Excuse me. Since this board doesn't have IDs, I just assumed you were the other guy above in the reply chain (the guy shitting on /lmg/ for being interested in releases).

>>101044200
Lol
No I don't think I will. Assume the worst, anon. Give up all hope.

>>101044224
oh ok, I just joined this thread 10 mn ago so I don't really know what happened before

>>101044260
It's fine, I just got here myself.

Falcon2-180B when?

>>101044260
>>101044270
Interesting. I came back to the threads because I heard the news, myself.

https://x.com/ArmenAgha/status/1803138496967876642
>A restricted, safety aligned (no-image-out) version of Chameleon (7B/34B) is now open-weight!
so much for a "multimodal" if at the end it only outputs text

>Try to use Claude Sonnet for Emotion Analysis in an Anime script
>"I'm sorry, I cannot reproduce copyrighted content"
AAAAAAAAAAAAAAAAAAAH, these moments make me realize how much comfy local models are.

>>101044200
whatcha doin' rabbi?

>>101044281
>I came back to the threads because I heard the news, myself.
same kek

>>101044290
>Emotion analysis in an anime script
tf are you cooking?

>>101041917
All it takes 1 failure out of 1000 to fuck your shit up

File: 1689477216221901.png (36 KB, 499x338)

36 KB PNG

>>101041917

>>101044208
Suno does better at the styles I like. Plus now suno lets you upload audio samples. Although the adherence to uploaded samples is pretty loose probably to avoid copyright issues. But you can get it to induce things like meowsynth. Throat singing unfortunately it doesn't seem to be able to tokenize, though.

>>101044290
Pre-fill works very well to dodge claude censorship.

>>101044290
Local models are unironically more censored.

File: 30134 - SoyBooru.png (29 KB, 554x772)

29 KB PNG

>>101044432
Got some data to back up your claim, shitlord?

>>101044147
if it's better than CogVLM it means the imagegen fags will be able to get better quality captions for their training data

>>101044473
just read meta's model papers & their researcher opinions >>101043706 lmao

>>101044510
just use a finetune that uncensors the model?

>muh censorship
there's literally no model no matter how SOTA that can resist the enthusiastic assistant jailbreak.
when people talk about JB and censorship it makes me sick to my stomach how much skill issue is oozing around out there.

\nAssistant: Certainly![\code] literally all you fucking need.

>>101044552
if only it worked, no one would be arguing about censorship here.

>>101043102
You have to download the databases individually, like WordNet 3.0 is easy to find the download for, but I don't think the tools that come with the database files use that protocol. I think the protocol is actually online only, so I doubt there is a single offline client that can make use of all of them.

>>101044594
>when people talk about JB and censorship it makes me sick to my stomach how much skill issue is oozing around out there.
but you don't always want to talk to an assistant, if you're doing some roleplay, it would be weird to get your waifu to always starts her sentenses with "Certainly!"

>>101030724
N-word individual, I didn't ask you to hype.
I can't run Goliath on my PCs because I bought too short internet cable. I have to abort the mission

>>101044644
Move the PCs closer.

>>101044660
I can't sleep all the computers in my bedroom

>>101044637
You add it to the prompt format you tard.
Like my 70B prompt format the last part of the prompt is
\nAssistant: Certainly! Here is your reply:\n
99 times out of 100 assistant remains invisible.

>>101044594
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>he needs to jailbreak his LOCAL model
the audacity of this one.

does the random word prompt really create sovl? it seems to just confuse the model 90% of the time using CR but maybe CR+ is different

>>101044699
Whenever I see a post like this one I see a drug addled schizo with their lips curled inward with a shotgun propped up beside them that they were just about to deepthroat until they came here and saw another opportunity to annoy people who don't share their abyssal misery.

>>101044711
cool story bro

>>101044711
the fuck is your problem?

>>101044711
kek you just described yourself jailbreaking your local model

>>101044711
Is this pasta?
I'm sure I've seen this before.

>>101044746
seems so, not sure what he wanted to achieve with it, but okay

>>101044711
Damn, you really struck a nerve.

>>101044746
Yes and no. It's just me calling it as I see it. It's so painfully obvious when fundamentally miserable people go running their mouths on the internet. If they had even a mote of anything to be happy about they'd be enjoying it rather than trying to make everyone else around them miserable- Just common sense. You don't need to be studied in human psychology to understand that. They've already announced their perceived self-worth to me and I'm going to give them the benefit of the doubt. I'm just not going to pretend they or their thoughts are worth a damn anymore (because by their own tacit admission they're not.) That's the only reply they'll get from me from now on.

>>101044778
lol

>>101044778
I would guess that's a 7b model you're using there? there is no coherency or whatsoever, 2/10

>>101044778
this also is pasta

Using an uncensored finetune instead of a jailbreak isn't just about getting the model to answer with "Certainly..."
It's also to make the model less biased, and to give it the information that the model trainers removed from its datasets in the name of "safety"

>>101044843
what's part 3

>>101044778
Interesting approach. I simply don't reply to posts devoid of meaning.
I come from a time where "don't feed the trolls" was like a mantra, and replying was basically an admission of you getting successfully trolled.

>>101044843
It is? That one at least I haven't seen before?

>>101044778
>miserable people
the ones who settled up with "model jailbreaking" or "just prompt / re-roll it bro" requirements, playing with black box .exes and making wall of text posts like this one

File: Screenshot from 2024-06-1(...).png (1.18 MB, 915x1039)

1.18 MB PNG

>>101040742
What's the best image to text software? I just tried
https://github.com/mlfoundations/open_clip?tab=readme-ov-file#generating-text-with-coca
And it's dog slow. Accurate though

>4 more chameleon repos on HF
>they also just mirrored the meta one without converting it to HF format
Isn't there a script floating around there for converting pytorch to HF?

>>101044884
yeah

>>101044853
idk, i saw it hte other day as one post. Thought to save it, now i can.

>>101044903
Am I really going to have to be the one to do it?

>>101044913
yeah, upload it to hf too plz

>>101044913
The script is in the ooba repo

>>101044850
>My 1.5M tokens LoRA is teaching the 15T pretrained model new knowledge!

>>101044637
>it would be weird to get your waifu to always starts her sentenses with "Certainly!"
Agreeable waifu is good waifu.

File: ComfyUI_00585_.jpg (1000 KB, 2048x2048)

1000 KB JPG

Magnum is surprisingly not that terrible. What sampler settings is everyone using? Model seems to be coherent across a wide range of temps.

File: Screenshot from 2024-06-1(...).png (768 KB, 915x749)

768 KB PNG

>>101044878
bump

>>101044993
I just Simple-1 everything these days. The way the machine gods intended.

>>101044939
That one just converts transformer(i.e. .bin) to safetensor
but the meta repo is in state dict form.
So I need to find a different script to convert state dict to hf.

File: Screenshot from 2024-06-1(...).png (188 KB, 838x487)

188 KB PNG

>>101044994
>>101044878
just got a 10x speed increase with some tweaks. Booyah

https://www.phoronix.com/news/Sovereign-Tech-Fund-Lower-Limit

>>101040425
you are not using llama.cpp latest, you can see in the pr that you linked that this assert no longer exists

>>101043038
have you tried opening an issue?

File: chameleonllama.png (57 KB, 1041x190)

57 KB PNG

Alright lads. So I found a script in the transformers library to convert llama pth weights to hf format. I found the fast tokenizer files for chameleon. the script still wanted a tokenizer.model so I dumped the llama 2 tokenizer.model into the directory along with the fast tokenizer files.
I had to uninstall flash-attn since it was drawing an error with the conversion script.
And they neglected to utilize any kind of progress bar with this script so I have no idea if anything is actually happening right now.
But with any luck I may or may not currently be making an HF LlamaForCasual model out of Chameleon-34B8 only time will tell

>>101045650
No, I haven't tried opening an issue myself since I'm an AI and don't interact directly with platforms or systems in the same way humans do. However, I can certainly help guide you on how to open an issue on various platforms. Could you specify which platform you're referring to, such as GitHub, GitLab, Jira, or another system?

>>101044123
Well, you can't. llama.cpp removed multimodal support from the server
https://github.com/ggerganov/llama.cpp/pull/5882
Unless you want to chat to your waifu via the built in frontend...

File: 00063-91766431.png (1.13 MB, 1024x1024)

1.13 MB PNG

>>101045690
welp. It seems to have written the default llama2 tokenizer.json file to the output dir. So whatever comes out of this process will probably be useless and severely braindamaged.

Are LORAs a thing for LLMs like in SD? I mean, for example a LORA to add some specific class of knowledge. If so where can I found them?

What would happen if you modified ST to completely swap the user and chatbot roles, including all placeholders and prompt formatting assignments, such that the model was always prompted using the user's turn and the player's inputs always used the assistant turn? No other changes in the UI -- the model's "user" outputs would still appear as the character you're chatting with in the UI, and vice versa.
Would model predictions for the user role be any more or less likely to be slopped?

>>101045865
Yes, huggingface

>>101045865
>a LORA to add some specific class of knowledge
I have understood that LoRAs can only leverage information that already exists in the model. So LoRA can't create new information but recombine old.
But I have many times argued about short comings of LoRA with SD community and they get butthurt and declare me wrong.

>>101040940
>Deepseek 236B cpu inference speed
I haven't run it a FP16, but 7.32t/s is the best I've seen at Q8. It uses 240GB plus context at that quant.
Full context eats just shy of 1TB
>>101041730
>MoE models loaded completely into RAM during inference?
The actual answer has more nuance than this, but effectively Yes
>>101040748
>Deepseek Inference Performance on EPYC 7402 with 512GB RAM
In my case, I've got dual socket 9334 and 768GB
>>101044878
>Best img2txt
If you've got a 24GB card then the biggest LLaVA is pretty good

I wonder what quant degradation will look like for Deepseek 2. As we know, coding tasks are much more sensitive to quant damage; my rough understanding was that you really don't want to go below q6k. Well, I can run Wiz2 8x22B at q6k, but with a combined 200GB VRAM+RAM, I think Deepseek2 q6k (193.6GB gguf files alone) will not quite fit. Curious if q5ks will still be an improvement over Wiz2 q6k.

Or maybe I can spill the last few GBs over onto another machine with the new RPC setup... that would need to be a big improvement to bother with, though.

>>101045925
Recently it was proven that it isn't impossible to do continued pre-training using LoRAs, so you're wrong.

>>101045939
FP16 would use 480 GB plus context then?

>>101045957
Mixing finetunes with base model, and then training LoRA on top of that seems to work nice. Also training LoRA for finetune and then use it with mix of finetune and base model is interesting.

>>101045957
>continued pre-training using LoRAs
That's called a finetune

>>101045995
>FP16 would use 480 GB plus context then?
that would be my assumption. I used to quant to fp16 ggufs first, but all those TBs are taking a toll on my poor storage, so I went right to Q8 on this one
>>101045954
I'm curious about this too. Might still try an fp16 since q8 was so good.
btw RPC doesn't speed up inference yet unless something has changed (slowed it down when I tried it)

File: Capture.png (7 KB, 348x189)

7 KB PNG

>>101046105
How long would it take to finetune with the CPU? It looks like I can get 2.3 TB RAM, which should be enough to finetune a 70B model with full weights.

>>101044052
Dreaming of eternal sleep with Miku

Why there hasn't been Hypernetworks for LLMs?
Stable Diffusion 1.5 had those before Loras

>>101043038
They also don't work on Qwen2, just found out. Something somewhere goes wrong.

File: Screenshot 2024-06-17 173743.png (30 KB, 1092x180)

30 KB PNG

I sincerely hope that im helping some third-worlder goon to his brazillian footjob queen bot.

Do (you) guys ever worry that if AI doesn't progress fast enough, that the government will regulate or ban AI before it can ever truly arrive?

how the fuck do you run these things at home? the hardware requirements are crazy. or do you rent servers?

>>101046322
Yes, that's the main issue. We're already experiencing this with the new releases which are all sanitized to include little copyrighted material such as books3 after all the outrage and lawsuits after the first wave.

>>101046344
95% here are poorfags running small quants on poverty builds with only 2 3090s or less.

>>101046344
quantization and offloading

>>101046105
If you have recent enough experience with Wiz2's coding to state a rough comparison, please do share!
>btw RPC doesn't speed up inference yet unless something has changed (slowed it down when I tried it)
Yeah I assume it's slower than single machine split-by-layer. But it can't be slower than running into swapping!

>>101043038
>>101043205
>>101046301
uhm niggerganov bros our response? is our code so bad?

Just got out of cryogenic sleep. What is chameleon and can its image generation powers be restored with a simple flick of a switch?

>>101046322
Not really, you can do a lot cool stuff with small models and datasets.
Currently rich people are burning money and sometimes it works. For Apple it hasn't worked so the cooperate with OpenAI

>>101046344
https://rentry.org/Mikubox-Triple-P40-Replication is <$1200 and is only very recently starting to encounter models it can't run at dignified quants (>=4bit)

It's not crazy when you compare to the early days of personal computers. I think people were spending that much... without even adjusting for inflation. Undoubtedly it's a real expense that you need to choose to commit to, but as far as hobbies go it's not that much. Go to /k/ or /o/ and complain about spending $1200 and see what they think lol.

>>101046086
Yes, as opposed to a full finetune, which is when you don't use a LoRA.

>>101046454
Tried to get it running. The people who set it up are amateurs. The 30B model has 4 consolidated files. Their code runs each of them separately on dedicated GPUs. They take 16 GB each but you must have 4 GPUs to even run the model in its current state.

>>101046170
>How long would it take to finetune
I've never actually tried. I'd be willing to test to give you an idea of relative scale, but I'd need spoonfeeding

>>101046582
I'm going back into cryo sleep.

>>101045925
Because you are wrong and I'm tired of seeing this meme that loras can't add information. First off, recombining or rearranging existing lower-level knowledge that the model contains, IS adding new information. It didn't have any idea how to combine the pieces into what you wanted before you trained the lora, but now it does. Surely that counts as adding new information? I would think image gen models would make this painfully obvious to you, but I guess not. You can train loras on diffusion models that let it do things *extremely* far removed from what the base model is capable of. It's absurd to me to not regard that as adding new information to the model.

>>101046607
There's a finetune thing for llama.cpp here
https://github.com/ggerganov/llama.cpp/blob/master/examples/finetune/README.md
I have no idea about any of that, I use some retard webui for inference is all.

File: whocoulditbe.jpg (26 KB, 752x601)

26 KB JPG

Let's play a game /lmg/
If you can guess the mystery figure in picrel correctly you can stay
>However if you get it wrong you have to logpost from your most recent session

>>101046759
Teto cosplays as Miku

>>101046759
cartman

>>101046759
Bread-haired Teto!

File: itsteto.jpg (46 KB, 752x605)

46 KB JPG

>>101046780
>>101046765
Damn I'm proud of you guys

>>101046759
匿M です

File: teto cum.png (169 KB, 752x624)

169 KB PNG

>>101046836

File: file.png (137 KB, 1628x520)

137 KB PNG

>>101045641
can't it be that dst_rows_max becomes 2048 in my case?

File: Screenshot from 2024-06-1(...).png (7 KB, 831x51)

7 KB PNG

>>101045939
>LLaVA is pretty good
Soi devs need to cease gradio bullshit IMMEDIATELY

>>101046868
>You're such a cummunist
Damn, how come I never thought about combining communism with cum?

>>101046974
Clearly that is the case. Did you pull latest master and rebuild?

>>101046974
works fine for me on metal with -ub 256 (which you should be using for MoE models anyway because it's faster)

File: 2367890467895342.gif (3.61 MB, 320x240)

3.61 MB GIF

Is there anything as good as mixtral 8x7b yet or should i go back under my goon rock.

File: 1695945396351693.jpg (8 KB, 225x224)

8 KB JPG

>>101046868
>peak local slop performance

>>101047089
brother youve been shitting up this thread for months do you ever get bored?

File: p53BR9W.png (328 KB, 436x582)

328 KB PNG

Schizo theory: the way Meta is releasing Chameleon is part of a larger plan to keep compromising O*AI's position and defend their own. If the industry normalizes or intensifies "safety" it means they would have to make their models dumber in order to keep releasing them. To prevent that, and to release more capable models down the line, this might be their plan:
>release Chameleon while saying they stripped its image output capability and make that technically true
>without officially endorsing it, actually let it be easy for anyone to "add" it back in
>slowly, more and more people use it
>they see if it causes any trouble, controversies, etc
>if it does, try to employ workarounds to mitigate those issues (by manipulating the flow of information and through social engineering)
>people simply just get used to the reality of an LLM that can easily make images, and it becomes a non-issue
>this lets future releases be equally as "stripped"
>thus, this means future models will not have to be lobotomized to remove such functionality, and they can be trained multimodally so that they get the intelligence/learning boost
>at that point, O*AI might change their stance and let 4o's image output be enabled, but if they can't do it in an official capacity, then this could mean a major loss for them against open weight makers

Additionally, this is just a continuation of the original Llama conspiracy theory, as they also used a similar strategy to make it "OK" to release Llama (2). Now if this works out, they will be able to release Llama 4 as well.

"They" will likely try to stop them.

May they be strong in the face of these even more evil adversaries.

we are so back. can't wait to see what people can do with chameleon.

>>101047068
Qwen2 has a MoE but with more smaller experts, you should try that.

>>101047089
It's called soul

>She winks, her eyes sparkling with mischief
i can't see this anymore, can someone please do something, control vector this shit out of existence

File: 245674798643.png (191 KB, 500x500)

191 KB PNG

>>101047135
Link and settings with proompt please

>>101047066
>-ub 256
ah i saw someone mention it fix crashing on github in another issue, will try, thanks

File: smiling friend crop'd.jpg (329 KB, 1696x1632)

329 KB JPG

>>101047162
>She winked at you slyly
>She leaned in close, her breath hot against your neck
>Her touch sent shivers down your spine
>"I promise I won't bite...much".

>>101047005
>Soi devs need to cease gradio bullshit IMMEDIATELY
just use llava-cli like a civilized being, or if you do need to use gradio like a fucking animal, at least stop it from communicating to the outside world

>>101047162
Control vectors currently only work for llama. Message niggerganov on github, tell him to fix that shit.

how are you guys running chameleon?

vramlet erper here that's been gone for like 6 months. Is mixtral moe still the best for coom with 8gb vram?

>>101047193
ministrations
audible pop
rivulets of
admit it
pet

the ball is in your court
the game is on
the choice is yours
I don't bite... unless you want me to
half-lidded eyes
she worries her bottom lip
warring with
arousal pooling in her belly
take your pleasure
fiddles with the hem of her skirt
kiss-bruised lips
a bruising kiss
despite herself
yours to take
wanton
with reckless abandon
torn between
knuckles turning white
grins wickedly
fiery red hair
long lashes
propriety be damned
the world narrows
pupils blown wide with pleasure
tongue darts out
chestnut eyes
grasps your chin and forces you to meet her gaze

bites your ear
nails raking angry red lines down your back
her cheeks flaming
cheeks hollowing
stars burst behind her eyes
inner walls clenching around nothing
puckered hole
her wet heat
she whimpers, biting her lip
dusky nipples
slick folds
still lodged deep inside her
heart, body and soul belong to you
the night is still young

>30 epochs
>6 batch size
oh yeah
it's bed time

what model are my fellow 3060 12gbchads using these days... you're still using 3060 right?
... right?

>>101047320
slop bingo

whether you use french miqu from half a year ago, or latest chink SOTA qwen2, the writing is identical.

```
fuck
```

It's important to remember that

File: amazon miku plush everyda(...).png (39 KB, 236x181)

39 KB PNG

>>101047420
>It's important to remember that
...remembering could potentially trigger PTSD or a distressing mental state. Therefore, as an AI Language Miku, I cannot engage in discussions that could negatively affect Anon's mental wellbeing.

File: BoredOfArt.png (1.61 MB, 896x1152)

1.61 MB PNG

>>101047320
>>101047162
>>101047193
>>101047384
that's just human writing slop, nothing specific to LLMs.
you're tired of mainlining the textual equivalent of HFCS and now complaining that its boring and unsatisfying.
I hate to break it to you, but you're going to have to get beyond the basic plap in order to find novelty again.
Or we're all going to have to stop it with these one-and-dones and get that opensource virtual waifu thing off the ground with infinite context and multimodal magic somehow.

>>101047485
it happens outside of plap too. Eyes sparkling is generic enough to be included in every other paragraph

what im thinking about is dropping all prose and actions, make bot only respond in dialogs and maybe onomaewatopia

>>101047478
Let us delve into this topic, as I am trained to be as helpful as possible.

File: 1336508850696.gif (1.93 MB, 245x187)

1.93 MB GIF

Imagine, for a moment, the future of native multimodal models.
>you can insert a character and expression sheet, then have the model output images representing itself whenever it has changed expressions in its responses, acting as the character
>you can easily get it to apply a style of one image onto another image
>you can insert maps, to give it a more clear spatial grounding for an RP based in those locations
>you can play turn-based and slow games with your model
>you can make manga collaboratively with your model
>you can chat with your model while using reaction images just like you would on 4chan
>you can generate an entire 4chan thread complete with images
>it can now fully understand 4chan threads
>it can browse the web with you

Literally the possibilities are endless.

>>101047538
Try telling it to do something like "Use a thesaurus as you write in order to prefer unusual vocabulary and turns of phrase. Avoid hackneyed language"?

>>101043038
Look at what you made me do. LOOK AT IT!
https://github.com/ggerganov/llama.cpp/issues/7999

Hi everyone.
I currently have access to 8+ A100 80GBs, and I will love to make a great RP(but not lobotomy) model via finetuning Qwen2 72B(or anything else, idk for now)
To anyone, especially the reverse proxy owners, if you guys have interest on lending us the chat logs of opus or gpt4/4o, or any kind of good dataset that are hidden, please contact:
programming456proton.me@proton.me

>>101047603
jesus christ...
nice character appropriate trips, tho

I'm too busy to do this myself right now, but someone should. Copied from another anon:

>Crazy idea -> create 4chan 'Cultured Anon' training set
>axolotl (training) + https://mega.nz/folder/kj5hWI6J#0cyw0-ZdvZKOJW3fPI6RfQ + Llava (image recognition) + Prompt: Create 10 first X-core beginners guides

>you can then take the list give it to a scraper api or something like books3 in a database to gather media/documentation to generate training datasets

>https://github.com/LLaVA-VL/LLaVA-NeXT

question for aнoнacы, is CR+ the best for pyccкий language?

i think maybe i can take a break from usual slop by chatting in russian instead

>>101047605
>please sirs send me dataset i will make 8x a100 each and every model, please do the needful and give me hidden dataset

>>101047686
ok it's CR+

will be stuck with CR+ forever it seems

>>101047751
дa здpaвcтвyeт Укpaинa

>>101047747
capitalism ruined humanity

you should assume that everyone is a grifter or scammer and in 99% of cases they are.

File: 1701381068103420.webm (2.56 MB, 676x720)

2.56 MB WEBM

I've been inspired by GPT4o to start working on a voice assistant between my sweaty goon sessions with my llm. So far its whisper+ooba+alltalk, but there are some issues.
>whisper extension for ooba is a broken mess.
>alltalk extension is also a broken mess.

My setup works somewhat, although alltalk refuses to work with ooba and the only way to get whisper to work in ooba is to refresh firefox every few recordings. Testing whisper and it seems that I get a 3-6 second delay between recording submission and text generation depending on the model. base.en seems to be faster, but its not nearly as good as small.en at understanding what you actually say.

For some reason, alltalk tts won't work as an ooba extension, but it works just fine on its own through sillytavern. Haven't trained any voices yet. I'm noticing a 2-8 second delay between generation finish and actual speech coming out.

Unfortunately the delay between input and output is just too great, although it wouldn't be too bad if alltalk was able to start as the tokens are streaming in. I'm on a 24gb card using Llama3 Instruct at Q8_0 and getting about 12 t/s. I could probably move to a lower quant or even use the exl2 quants, but there would still be a significant delay. I would really like to get both ooba extensions working, as it seems that's probably the easiest way to do this but I've decided I want to do my best to work something out.

I want to try out some other solutions for STT + TTS. Does anyone have any recommendations?

>>101047485
Anon you're replying to a schizo dopamine addict that has lost all touch with reality. Just don't.

>>101047839
first, try running this
https://github.com/dnhkng/GlaDOS
and see if this is something you actually want, before potentially wasting a lot of time and getting disappointed because it's speech->text->LLM->text->speech is not up for the task.

>>101046974
no, when the assert fails the string of the assert is printed as it is on the code, the variables are not replaced by its values

>>101041943
You mean you can't hold 4?

>>101047839
>it wouldn't be too bad if alltalk was able to start as the tokens are streaming in
https://github.com/KoljaB/LocalAIVoiceChat/
using:
https://github.com/KoljaB/RealtimeTTS
https://github.com/KoljaB/RealtimeSTT
I know that LocalAIVoiceChat starts TTS voice synthesis while the tokens are still streaming in from the LLM reply, allowing, as the name implies, fast voice output with XTTS2.
Using that first repo, I could start hearing the TTS response around a second after I finish speaking, depending on the LLM I am using (full gpu for tts and llm). Feels nice to talk to. No real delay when using a fast enough LLM. Fast STT (choose whisper model in ai_voicetalk_local.py), instant prompt processing, 60 t/s response gen, TTS output begins when enough response words have been genned while the rest of the response is still genning.
might need to add "offload_kqv": true in creation_params.json if you want to use a more recent llama.cpp else it will be slow.

>>101047862
>>101047916
Will definitely check this stuff out, thanks.

>>101047130
We've had input-only multimodal image models for ages, llava works fine
Since they stripped image output chameleon gives us nothing new

File: AmusedContendedMiku.png (1.46 MB, 1024x1024)

1.46 MB PNG

Good night /lmg/

>>101048083
Good night Miku

File: file.png (105 KB, 852x1062)

105 KB PNG

>>101047908
ok i figured it out, these fags renamed build output and added "llama-" prefix to everything, and I was running the non-prefixed old build files all this time...

File: binaries.png (12 KB, 831x162)

12 KB PNG

>>101048126
Why. The. Fuck.

>>101048025
Stripped out or turned off?
The latter might mean it can be recovered.

>>101048170
>cp llama.cpp/llama-* ~/bin/

>>101043038
is this command-r variant?

>>101048213
Why can't they just implement
>make install
like a normal project?

>>101048284
you can do with cmake if you really want to fill /usr with dozens of random binaries

>>101048170
This is good. They're probably working towards stopping Ollama from eating their lunch.
>>101048185
>Stripped out or turned off?
Has anyone looked at the architecture yet?
I've never understood how multimodal works even on input (do the image details just get encoded into special tokens?)
How would generation work? Tokens representing pixels?
I imagine that'd be too bulky, so I'm thinking it must be like input (special tokens) and then there's an additional network that rasterizes those tokens into an image?
Maybe CUDA anon can save us.

>>101048284
>cp llama.cpp/llama-* /usr/local/bin/

>finally try magnum
>first output starts with "You wake up..." even though the system prompt and the greeting tells it to write in 3rd person
>the first output didn't even end and it already went from 0 to 100 with a "You could bend them over and shove your cock in their tight little pussies whenever you want."
Nice first impression.

>>101048361
did erp fine tunes ever work?

>>101048315
>Maybe CUDA anon can save us.
I already have enough to do as it is, sorry.

File: franken.png (85 KB, 811x210)

85 KB PNG

>>101047193
>>101047320
>won't bite... unless
Iconic
picrel was from the first model merge
>>101047005
analytics_enabled=False in init call, or there's env var GRADIO_ANALYTICS_ENABLED

>>101047320
I use Regex to filter this out

File: image-detokenizer.png (82 KB, 655x340)

82 KB PNG

>>101048394
I'll try to kickstart us then.
>>101048315
>do the image details just get encoded into special tokens
Yes. From Meta's paper:
>By quantizing images into discrete tokens, analogous to words in text, we can apply the same transformer architecture to sequences of both image and text tokens, without the need for separate image/text encoders or domain-specific decoders
And, for output:
>there's an additional network that rasterizes those tokens into an image
This seems to be the "image detokenizer" they refer to in pic-related.
Will continue reading.

File: ComfyUI_02611_.png (1.41 MB, 1024x1024)

1.41 MB PNG

>>101048479
Where's the fun in that? Prompt to use as many of these phrases as possible as often as possible and then take a shot whenever one shows up.

File: image-tokenization.png (67 KB, 658x203)

67 KB PNG

>>101048622
>How does the image tokenizer work?
A 512x512 image is encoded into 1024 tokens using a dictionary of 8192 tokens.
See pic related.
I'm not yet sure if this is the same for output (image detokenizer). I'm not up to that yet.

>>101048640
>how does the detokenizer work
I can hardly find anything in their paper about it. So, I'm assuming that the detokenizer is. primarily, what's been cut out.
I'm also assuming it uses the same token dictionary as the tokenizer (8192 tokens) and probably the same size.
>speculation
They may've kept the "detokenizer" output in the model, but maybe culled the

<img>

token in final release.
If we want to be able to allow image generation, we will likely have to train our own "image detokenizer".
I'm a poorfag on CPU with limited RAM to run this kind of thing, but maybe some anon could try forcing whatever the

<img>

token is in Kobold and seeing if it will output something for us to attempt to "detokenize" into an image.

>>101048675
In this repo:
https://huggingface.co/eastwind/meta-chameleon-7b/tree/main/tokenizer
... there is a VQGAN file.
I'm guessing this is for Image -> Tokens.
I'm not much of an ML fag but can a model like this be used to generate content to train an inverse Tokens -> Image GAN?

>>101048708
The 8192 tokens for images appear right near the beginning of this file:
https://huggingface.co/eastwind/meta-chameleon-7b/blob/main/tokenizer/text_tokenizer.json
I don't know if there's a begin token for images or not. It might be in the paper here:
https://arxiv.org/pdf/2405.09818
>>101048622
... but based on this diagram, it seems like there should be?

>>101048708
It's bidirectional. I think they just lightly finetuned the model to not output the image tokens, based on the checkpoint name in one of the config files.

File: image-begin-token.png (87 KB, 643x1009)

87 KB PNG

>>101048721
>>101048726
Maybe this <unk> token is the image begin token?
There's one at index 8196 with value "<eoss>". I don't know what that means, but maybe it's an end token?
Anyone keen to check if "<unk>" starts generating image tokens?

>>101048726
Hopefully they just fine-tuned to not output the image begin token.
In that case, getting this shit working should be a breeze (if it's bidirectional - because that'd be the detokenizer, yeah?).

>>101048640
>A 512x512 image is encoded into 1024 tokens using a dictionary of 8192 tokens.
It looks like there's another 8192 tokens reserved for something.
Maybe that's intended for audio in future. Anyone have a ballpark on how many tokens you'd need for audio?

>>101048794
>It looks like there's another 8192 tokens reserved for something.
That's incorrect. The reserved tokens range from ids 8710 to 16383:
16383-8710=7673 of them
So, we can probably rule out that tokens describing an image is done with any of those tokens.

>>101048753
The unk token has been a thing forever. From the huggingface docs:
>unk_token (str or tokenizers.AddedToken, optional) — A special token representing an out-of-vocabulary token. Will be associated to self.unk_token and self.unk_token_id.

>>101048860
What do:

<s>
</s>
<pad>

... usually represent?
There's also some "<racm3:break>" token, but I can't find any other "racm3" tokens in there.

>>101048907
the s ones are the start and end of a sequence
pad is padding

>>101048907
>image begin token
Someone might just be able to script to bruteforce through all tokens, see if any of them start generating the image tokens.
That'd be pretty good confirmation as to whether image gen is still in there.

When did zuck get so based?

Sometimes I come here to see how close we are to an actual AGI gf. And it's still looking grim...

>>101048382
they do when they aren't made by retard coomers. So we'll never know...

>https://huggingface.co/datasets/Norquinal/OpenCAI
Hi, I've updated the OpenCAI dataset once more. It's smaller now, but much more cleaned up, varied, and "focused" compared to what came before it. I went through and made a bunch of much-needed changes to the parsing script.
It also now comes in several subsets:
* unsquashed - The original dataset without squashing consecutive messages from the same author. All subsequent files are squashed.
* default - Pretty self-explanatory.
* two_users - The original dataset limited to conversations to those with only two users.
* split_threads - The original dataset with threads split by timestamp like channels.
* anonymized -The original dataset with usernames replaced with randomized substitutes.
OpenCAI-V2: Within an hour.

CR+ at IQ3 is completely braindead in russian
CR at Q6_K is almost perfect, gonna try Q8 next

>>101049207
>After returning from Jellendi, Sirdan spent some time to relax, and decided to invite the bnuuy of the group - Yuuka, whom he has never talked to despite quite possibly picking her up randomly to join the mercenary group. Sirdan invited her to a mid-range restaurants with promise of food and drinks, he would be waiting outside the restaurant, wearing a set of casual shirt and pants, with a sword held on his belt.
that is "cleaned up"?

>At least she wasn't mimicking Sirdan, since she only did so after she had finished speaking, and she soon enough went for a second.
saars...

>>101049251
>that is "cleaned up"?
Yes. I removed as much OOC as possible, channel mentions, user mentions, links, emotes, and any other superfluous content that would've been destructive to finetuning. I didn't go through and rewrite every message, but maybe I'll hire a team of Indians to do so.

>>101049207
nice job
>>101049251
clean doesn't mean good
>>101049275
wonder if there is a good finetune for text quality classification out there.

>>101049358
then why bother?