File: threadrecap.png (1.48 MB, 1536x1536)

1.48 MB PNG

►Recent Highlights from the Previous Thread: >>102535977

--Paper: Improving code generation in large language models:
>102541159 >102541640
--Papers:
>102542180 >102542337 >102542532
--Experimenting with high dropout rates and fine-tuning:
>102542430 >102542447 >102542469 >102542583
--Finetune data requirements discussion:
>102536472 >102536511 >102536539
--Enabling Model scopes for Vector Storage did not regenerate vectors:
>102537930
--Discussion on the potential and performance of distilling 70B-instruct to 51B:
>102536804 >102536822 >102536823 >102536885 >102537261
--BUERgence script for optimizing llama.cpp inference parameters:
>102542933
--Approaching 16k context limit and potential solutions:
>102542492 >102542636 >102542733 >102542746 >102542830 >102542851 >102542865 >102543206 >102543451 >102543037 >102543295 >102543426 >102544540 >102544569 >102544739
--Advanced Voice rolling out to ChatGPT Plus and Team users:
>102537776
--45 minute daily limit on advanced voice features, including silence:
>102541866 >102543472 >102543491 >102543519 >102543677 >102543759
--Miku (free space):
>102535995 >102536036 >102536082 >102536179 >102536277 >102537490 >102538991 >102539843 >102540239 >102540324 >102540465 >102543667 >102544553 >102544704

►Recent Highlight Posts from the Previous Thread: >>102535999

Do people here use agents why/why not?

>>102545041
"agents" aren't a coherent concept and everybody who talks about them sounds like one of the crypto grifters who pivoted to ai

>>102544853
most useless post itt

>>102544853
seriously, where's a script to fix these backlinks?

>>102545041
Generation and especially prompt processing is too slow for building agents, while smaller models require fine-tuning to function reliably. Time isn't a factor in only a few tasks.

>>102545060
I understand "agents" as any indirect use of LLM output, or more specifically, instances where output results from additional automated prompts.

>>102545116
Errors are snowballing if you use an LLM output as input for another LLM.

>>102545060
I mostly meant stuff like this
https://github.com/Agent-Tools/awesome-autonomous-web

File: 1727226087946704.png (489 KB, 512x768)

489 KB PNG

>>102545137
Just like any LLM response, since each following token depends on its preceding ones. Agents offer a way to enhance answers by focusing on specific tasks. For instance, you could employ an extra prompt to track a character's clothes or outline a broad narrative plan beforehand. It outperforms a single prompt, but waiting for subsequent prompts to finish before final answer even start streaming is not worth it.

>>102545307
You can do the agent shit while the human is writing out their response. Assuming that order is acceptable.
Human response
AI response
Agent1 gen: {{char}} clothing state
Agent2 gen: {{user}} clothing state
…
(Eventually)
Human response (suffixes above state info before ai response optimally wrapping into fancy prompt)
(Optional: do judge rounds on the above answers to improve quality, if the human is taking his time responding)

File: 1415322611803.png (5 KB, 208x208)

5 KB PNG

I love LLMs so much, bros. When I tried AID2 back then, I had a clear vision of where it would go for entertainment, and I feel like I'm living in that vision once I finally achieved 70 beak models locally, specifically Q4 or higher. The ability to follow rules leads to so many opportunities - text adventure, games featuring {{role:d20}} against {{role:d20}}, roleplaying with wild traits mentioned as just minor notes, and so much more. And things I hadn't even considered, like an offline personal wikipedia or browser search for information, recipes, questions, or a writing assistant for professional papers. The only front I feel it's still lacking in is fanfiction. It's certainly more coherent trying to simulate established characters, but 70B or modern finetuning is still too rough to excel at it.

This tech has been a novelty for me for years, but with that last hardware upgrade it became integral to my life. Local is definitely the future, and that future feels bright. I can't wait to see what the next jump 5 years from now is like.

>>102545367
buy a muffin because you deserve one

>>102545307
Maybe you could get away with a small dumb tinyllama for that

File: 16lines.png (120 KB, 685x374)

120 KB PNG

i had an idea tonight about using ai training to store data. the idea is simple, create small images of hex text files, then square crop the image and use it to train ai. the gpt can already convert hex values but thats not wat i want.

i guess the desire is for the hex charts to be stored as training data. then recalled by GAN would make a new text file? this could be used for raw data eventually? ai generated bitmaps?

pix related, concept would crop off the text and just have a square of hex.

>>102545367
>The only front I feel it's still lacking in is fanfiction. It's certainly more coherent trying to simulate established characters, but 70B or modern finetuning is still too rough to excel at it.
You should try to use a lorebook for world-building and these fine details.
>I can't wait to see what the next jump 5 years from now is like.
I don't.

>>102545442
>using ai training to store data
Retarded idea.

>>102544853
where is the recap image?

>>102545503
It didn't pass our safety checks

>>102545440
You may achieve good results with proper task-specific fine-tuning.

>>102545486
>nooo don't use a compression algorithm for compression
Calm down silly anon.

>>102545442
>GAN
you can probably get away with a simple deconvolutional NN with the image ID encoded as a vector as its input

File: hughjackman.png (244 KB, 422x506)

244 KB PNG

So is Mistral Large 2 still the go-to model for around 100B? Is 4 bpw enough or is 5 bpw necessary for it not to shit itself?

>>102545307
>subsequent prompts to finish before final answer even start streaming is not worth it.
I was thinking about it a few threads ago but never said it out loud? Does batching need more vram? Cause you could generate initial draft and then do agent stuff in parallel? Originally my idea was not related to agents but just running multiple gens for the same prompt with different seeds. Not worth it for 7B etc but if you are running a 70B having it write 5 different drafts at the same time with same speed would be nice?

File: recap-102544848.jpg (2.67 MB, 1253x7330)

2.67 MB JPG

>>102545078
https://rentry.org/lmg-recap-script
>>102545503
I figured I would stop with the images now that we have the script.

File: 16lines_crop.png (81 KB, 420x323)

81 KB PNG

>>102545538
yes this is how the thought had started, could ai train'd data take up less space @ scale (thousands of hex images). since it's redundant and based on calculus weights, gradient decent etc.

>>102545592
i'll keep that in mind if i can try but it's vaporware imo since i dont know how script any models except for easy-difussion, which is already trained on ten gigs of images.

>>102545486
say it was trained on the kjv bible in hex blocks, the ask for an output image of what was learned, then run an ocr to get the hex values back, and decode to text.. what would u get?
> in heaven there is no beer, that's why we drink it hear !
who knows what might come out. assuming the hex couplets were'd lost into a mass of scrabbles. we take OCR training for granted after all...

>>102545694
>https://rentry.org/lmg-recap-script
thanks
now how do I make them work with 4chanx previews

>>102545735
>now how do I make them work with 4chanx previews
4chanx previews work with the user script.

>>102545760
they definitely don't
I'm also not getting any (cross-thread), (you), or (dead)

File: 20240925_181923.png (106 KB, 1344x938)

106 KB PNG

>let's have the AI think for 3 months and cure cancer bro

all models below 70B are shit. I wanted to create an RPG game that uses LLM. But most local models below 70B are not suitable, often do not follow the instructions, or start to get lost. Few people can run 70B locally on a computer. APIs such as openrouter/groq are expensive or very limited for a simple game. So games using LLM are still the future. If models with BitNet (or another revolution) do not start appearing, such games with LLM will not be common. Unless special models 8B are created that are trained in terms of using them in RPG games.

>>102545841
You people said this exact same thing a year ago when the best we had was llama2 70b which is worse than llama3-8b.
Admit it, you only care about justifying the money wasted on gpus.

>>102545878
>You people said this exact same thing a year ago when the best we had was llama2 70b which is worse than llama3-8b.
Google Gemini is currently the best 8B on the market and it beats all old 70Bs, so yes, small models improve rapidly, but so do big models.

>Admit it, you only care about justifying the money wasted on gpus.
No, we are not. It's about getting accustomed to the best thing. Models just feel dumber when you go down. Once you taste the best, you don't want to use anything less.

Ok, L3.1-70B-Hanami-x1 is the first good 3.1 tune imo. Follows instructions better than anything I've ever used before and writes extremely well.

>>102545878
Oy vey :O

>>102546005
Hi, Hanami.

>>102546005
good morning saao

>>102545442
I don't get it. You want to train on the image of the hex? The raw text would be just 256 bytes (plus some EOT token or something), while the image would take at least 11288 bytes assuming a 6x8 character font. Even at 1bit per pixel you still have 1536 bytes to feed to the thing. You're increasing the training pressure by 6x at least.
If you want to compress data there's already llama-zip and there's a similar thing as a PR on llama.cpp already. A classical compression algo would be faster and more reliable.
If you want to add noise to the generation to get something like
> in heaven there is no beer, that's why we drink it hear !
You use token masking during training. You can even train to include typos.

>>102545878
>llama2 70b which is worse than llama3-8b
Would you prefer l2 or l3 to suck your dick?

>download mistral nemo 12b
>chats no longer devolving to "YES MARK ME AS YOUR ____"
>get into a nice chat with ai, just a normal interaction
>things getting kissy
>"by the way, [user], before we do anything i have to tell you something
>"im from a different dimension and blah blah"
where the fug did that come from? card mentions nothing about secret past or hidden origins and nothing in the chat was magical/otherworldly/interdimensional. i tried regenerate and it got weirder, one time she said she was a demon kicked out of hell, another try she said she could "travel through time and manipulate energy" lel

or is that crossing the context limit (16k)

>>102545841
>APIs such as openrouter/groq are expensive
70b is like $.40/Mtok
most of the bigger models are more expensive but 70b is dirt cheap relatively

>>102546781
Honestly anon
I think you're just cursed

>>102546781
maybe a temperature thing
i think recommended temp on non-merge nemo is something ridiculously low like 0.3

>>102546809
i'm starting to think so as well
>>102546814
ah i'll have to try taht

>>102546781
Have you posted your settings yet?
Because that sounds like a case of bad settings + some weird shit somewhere in the context.

>>102546781
nemo instruct is very sensitive to things like whitespaces in the instruct template, I switch between two templates depending on whether I want creativity or accurate instruction following

>>102545728
>could ai train'd data take up less space
This is as retarded as people thinking PiFS is viable. Spoiler alert: it's not. The offsets will, on average, take more space than the shit you're trying to "store" in it.
Same with this, how the fuck are you going to store data in a NN without any losses? If you think for at least 1 second you'll realize that the absolute best you can do is a 1:1 space efficiency, and that's after training and inference.

>>102545783
>we have 10 datapoints
>now we can make a parabola with it that will model the accuracy!

>>102545783
>Only two datapoints in the first thousand tokens of the test
This shit sucks.

File: file.png (124 KB, 1051x530)

124 KB PNG

>>102545777
It works for me using 4chan-x and GreaseMonkey. Are you using xt or one of the other monkeys?

>chat completion
utterly censored
>text completion
utterly uncensored
why?

>>102546891
Probably because you aren't using the exact instruct format that triggers that behavior when using text completion.

>>102546891
Because raw smut is easy to find and train with, even accidentally. For a big dataset collected from the web, there will be some. For chat completion they typically generate the training set, so they'd need to generate smut requests and responses and nobody's gonna want to do that job. Not at any big company. It's easier to just say "no, i don't do that" and leave it at that.
I assume you mean the source models, not the finetunes and merges.

>>102546888
yeah, tampermonkey, guess I'll die

>>102546956
Qwen2.5 32b instruct, but I think its because of the default instruction template, I'll see what happens by tweaking it

>>102545841
Im also experimenting with llm npc and so far its a disaster. It is most likely due to the lack of situational awareness of the npc.
Lets say you use an llm to act as the guard of a gate that is closed and there is no way the player can get inside. The player can prompt the AI with things like 'i climb the fence and go inside' the ai will respond like if the player really did that when in reality the player is standing outside the gate. This is completely immersion breaking and the restrictions and workarounds i've tried haven't worked so far. Maybe with ai that can undertand pictures you will be able to send the ai the screencap of the situation plus a description plus the user prompt, thus creating a proper answer.

>>102547036
I think that, for that kind of interaction, you sort of need a multiprompt solution.
Something like prompting the model for the current context the player is in and if what the player wants to do is possible, what are the possible consequences, etc.
Iterate on the player's prompt before outputting a response, basically.

>>102546979
If it was trained to reject smut, and i understand it was, chances are that tokens exclusively found in the chat template will steer it towards safe outputs. You may get better results, but those tokens have been burned into the model. There's no context where the chat template tokens and smut overlap. It's like having any mention of 'assistant' in the system prompt in llama models.
Best of luck, though.

smedrins

>>102547079
That's an old word, Anon.

>>102547077
>tokens exclusively found in the chat template will steer it towards safe outputs
I was somewhat suspecting that was a thing, that the chat template would be utterly different from the regular text completion, oh well =(

>>102547055
I've thought about this too, embed the player prompt with a complete situation description and if the actions the player is describing are legal or not. Would require various iterations.
Is qwen2.5 image understading any good?
Also all of this should be done by a model of no more than 11b

>>102547186
Ideally, you'd have a whole game built around the llm instead of trying to have the ai run the whole simulation.
So inventory management, calculations, etc, would be done by a classical system, then the llm would be fed that information, alongside the relevant history of its previous outputs and user's messages (summarizex RAG or something of the sort) then it would evaluate the user's input before producing the final output (iteration).
I wonder if the better approach would be to have a fast small llm that dopes more iterations or a bigger slower but more capable and "stable" llm with less iterations.

>>102547133
Nothing stops you from trying, but i wouldn't expect a night and day difference. May work at the start but as the context is filled with 'safe' tokens, so will the output.
Text completion has no template, and you can use an instruct/chat model as a completion model. The problem is that anything that steers it a bit towards the 'assistant mode' will cascade and start rejecting things.
I remember, back when i just started using llms, using instruct models as completion with few-shot examples.

Some setup.

char1: dialog...
anon: dialog...
char1: dialog...
anon: 

Set "\nchar1:" as the input suffix string, set "\nanon:" as the reverse prompt and that's pretty much it. You still get to dialog with the thing but, hopefully, avoid all the steering tokens. You can add new characters on the fly, you could even play as more than one by adding more reverse prompts.
I don't know if it worked well because of that token avoidance or because i was using a model that didn't care about those things anyway. Never had an out of character rejection (good guy would still refuse to kill another person, while a bad one wouldn't).

File: file.png (36 KB, 666x370)

36 KB PNG

>>102546961
Try this. Go to the settings, and set it to Run at: document-body. I installed TamperMonkey and that got the previews to load for me.

https://molmo.allenai.org/blog

>>102547325
yeah that works, it was either document-body or a position above 4chanx

>>102547425
Buy an ad

>>102547425
>MolmoE-1B is a multimodal Mixture-of-Experts LLM with 1.5B active and 7.2B total parameters based on OLMoE-1B-7B-0924.
Interesting.

>>102547425
looks cool
i'll play with the 7b when a quant drops

File: zs.png (43 KB, 411x184)

43 KB PNG

>>102547425
based

>>102547242
i would take the approach of having a small model(2b-3b) doing the understanding of the situation before feeding it to a bigger model(7b-11b).

>>102544853
this is just spam now

File: VHDO2 script.png (43 KB, 790x786)

43 KB PNG

More whacky results from experimenting with super high dropout training.

File: sorry.png (35 KB, 394x629)

35 KB PNG

>>102547870
AGI achieved.

so what do I use for important matrix ,
just the dataset used for training?
or
could I use a series of questions that are likely to trigger the parts of the model in need to be preserved? would just the questions or question and partial answers work ?

>>102547425
Is their demo running the 7B or something? The results I'm getting from it are far below what I'd expect from a 70b model.

File: molmo7BD.png (49 KB, 896x140)

49 KB PNG

>>102548005
>Is their demo running the 7B or something?
Yes. Molmo 7B-D

>>102545841
>But most local models below 70B are not suitable, often do not follow the instructions, or start to get lost
There are improvements on this front as you get more beaks, to the point where somewhere between deepseek and 405b they are smart enough to be a competent dungeonmaster for you with the right system prompt
>such games with LLM will not be common
NVidia is gatekeeping hard right now, but eventually consumer hardware that can run quadrillion parameter models will be cheap and common.
I have seen the future, and it is fun

>>102548005
yes
>Select model weights, inference code, and a public demo (using Molmo-7B-D model) are available starting today.
I don't know why they wouldn't demo with the big one. guess costs, but it makes a bad impression especially when they don't make it very clear what model is running in the demo

>>102547425
Personally I don't see a use case for multimodal models. As long as my eyes and ears are still working. And multimodal models are much more retarded than text-only.

>>102547955
Read
>https://github.com/ggerganov/llama.cpp/discussions/5263

>>102548038
>but eventually consumer hardware that can run quadrillion parameter models will be cheap and common
not with semi conductors

>>102548066
>not with semi conductors
whelp, back to analogue computers then

>>102547955
>>102548057
Shit, I meant to also link
>https://github.com/ggerganov/llama.cpp/files/14194570/groups_merged.txt
Which is part of that discussion.
There's also some cool information regarding the process itself.

>>102548086
There's some research on analogue computers built with light pulses that mimick the behavior of neural nets. It's super interesting.

>>102548030
>>102548045
Okay, good to know. It's definitely dumb to showcase the 7B like this without making it very clear in the demo. I was more than ready to disregard the entire model family as trash.

>>102546867
You could fit a parabola to 8 data points and get perfectly usable results.
All you would have to do is correctly estimate the input uncertainties and calculate chi2/NDF (and also check that you can use this metric).
But the extrapolation of these results would be extremely sketchy no matte what because the uncertainty per data point is obviously very large vs. the change in "true values".

>>102548057
what's pseudo-random synthetic data mean In the context of that conversation?
is it the text equivalent of regularization images?

>>102548086
there's quantum computing
however due to the nature of the wave function there's a chance that it may never be technologically feasible for it to be useful .

>>102548144
>what's pseudo-random synthetic data mean In the context of that conversation?
I think it's data generated by another model (or the same model I don't know) that's randomly sampled to create a dataset.
I think kalo's general calibration data is randomly sampled fragments from The Pile.

File: 1698516170440216.png (130 KB, 862x640)

130 KB PNG

>>102547425
>Countbench 91.2
Damn, they absolutely maxx'd on this one

>>102545680
5bpw is where the loss of intelligence becomes hard to spot, but 4bpw is still good.

>>102547425
How do you run this? Their trailers imply that there'll be some fancy front end that shows how it counts people and stuff but there's nothing about a front end on their roadmap.
Local models are very held back by the fact that all our front ends suck dick, especially when it comes to multi-modal shit.

>>102548191
but what does randomly sampled here mean?
just samples from a larger dataset?
it's response to noise/incoherent text?

File: file.png (46 KB, 889x853)

46 KB PNG

>>102547900
soulful

File: demo_motion_breakdancing-(...).webm (685 KB, 1920x748)

685 KB WEBM

Have you guys seen this? Maybe it'll be local, that would be cool
https://menyifang.github.io/projects/MIMO/index.html

>>102548365
Nice, finally something for the vtubers

To the fine tuners in the thread (sao, drummer, whatever) I propose an experiment.
>get the recipe for a nemo 12b fine tune (Ie. Lyrav4)
>prune mistral-small using https://github.com/arcee-ai/PruneMe down to 12B~ish parameters
>fine tune the resulting model with the exact same recipe and data as the nemo fine tune
Maybe add a "healing fine tune" using the model's own data before the last step.
I'm curious to know how the final models would compare.

>>102548324
>but what does randomly sampled here mean?
>just samples from a larger dataset?
If I'm reading the exchange correctly, yeah.
As jukofyork mentioned in his proposal:
>It's likely the use of random and semi-random data mentioned in this thread is acting as a "quick and dirty" form of regularisation anyway:
>https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/bishop-tikhonov-nc-95.pdf
>and IMO, it would probably be better to consider doing it in a more principled way - especially considering the calibration dataset is so small and the imatrix computation isn't using the full context nor the correct prompt format, etc.

>>102548086
You'll have to train every instance of the model from scratch with analogue (just like nature does it.)
Big tech wants quickly reproducable slaves, and nvidia is all in on semis, so we're stuck with emulation and toy models.

File: dropoutrate.png (51 KB, 711x272)

51 KB PNG

>>102547870
For LLMs and LoRA finetuning, the optimal dropout rate is in the 0.55-0.60 range. https://arxiv.org/pdf/2404.09610

With higher values you get increasingly higher and more frequent grad norm spikes / noise that disrupts the training, as well as reducing the network's capacity for learning (at 90% dropout, every weight update is only updating 10% of the moderl/LoRA's weights).

>>102548570
most people use between 0.05 and 0.1
I'm surprised nobody is applying any of this high dropout rate research to any finetuning. It definitely integrates the new behaviors a lot better albeit my experimental results take it to the extreme.
I might try tuning a base model with an RP dataset using high dropout methods in the near future though. Obviously using it on instruct models is kind of hit and miss since there's already an existing behavior that permeates the entire model.

>>102548715
I'd take it with a grain of salt. The paper is from the alpaca dataset days. That dataset was extremely repetitive with some questions like, "sort this array" appearing hundreds of times. High dropout would help there, but might hurt for more complex datasets.

>>102547425
>Broadly speaking, the academic benchmark results and human evaluation strongly agree, with the exception of Qwen VL2 which performs strongly on the academic benchmarks and comparatively under performs in the human evaluation.
oh no no qwenbros...

File: PerpetuallyHappyMiku.png (1.34 MB, 800x1248)

1.34 MB PNG

good morning /lmg/!

>>102548761
I tune at home so failed experiments only cost me a dollar worth of electricity, so I'm not too worried. But I thank you for the handy graph. Next time I'll try 0.6 (I've done 0.75 and 0.8 so far, both using raw text datasets) But as always I'll post the results here when I do my next experiment if they are interesting at least.
Probably going to do a qwen2.5-7B RP model next.

>>102547809
It is a spam that is avoiding the anti spam filter.

>>102548030
>compares favorably to gpt 4, claude 3.5
>7B
>actually not even a 7B
??? what?

>Imagine you're trying to find the minimum of a hilly landscape. A high β1 (strong momentum) would make you "roll" down the hill quickly, but you might overshoot the minimum. A lower β1 would make you "walk" more carefully, potentially finding the minimum with greater accuracy.
guess the model

>>102545694
>https://rentry.org/lmg-recap-script
inline this link in the recap itself in the future?

File: its8exclamationmark.png (40 KB, 1261x172)

40 KB PNG

>>102548960
That's just how they named it. I assume it's to put it in a familiar range that has existed for a while.
Unless you mean something else. Complete sentences make everything clearer.

File: 1723844433490791.png (61 KB, 1181x185)

61 KB PNG

>>102548960
>>102549019
god i hope this kind of naming faggotry doesn't become commonplace in the ai industry

>>102549069
misleading marketing and retarded naming schemes are already the norm in the ai industry

>>102549094
>>102549069
>>102549019
Normal in most industries.

>>102549069
Molmo 1B (1 bazzillion nibbles) looks pretty good.
But it's still more telling than -small or -large. 1B params at that scale doesn't make that much of a difference.

>>102549105
I'm not complaining. I just clarified a vague comment made by anon.

A year later, llama.cpp might add a jinja parser.
https://github.com/ggerganov/llama.cpp/pull/9639
Maybe, llama.cpp will be usable by itself in 2025.

>>102549141
Bloat. The template format could be specified clearly for every model and we wouldn't need anything other than --in-preffix and --in-suffix.

Bros... Molmo-7B-D seems really really good for captioning. I integrated it into my captioning scripts and have been testing it on my dataset of women peeing (don't ask...). It seems to perform roughly on par with InternVL-40B. Just as uncensored, will describe nudity, visibility of breasts and genitals. Slightly worse than InternVL at accurately describing the pose, and some details of the clothing. But much more accurate and consistent at describing that the woman is peeing (so many VLMs just completely cannot "see" that aspect of these images). Will try to get the 72B running locally soon and see how much better it is. I generally consider InternVL-40B to be the best local captioning model, and even the 7B of molmo might be the new king.

>>102549204
Psyoping us with really bad fake shills so the "buy an ad" posting seems more warranted isn't gonna work.

File: mark-zuckerberg_gettyimag(...).jpg (274 KB, 1200x1200)

274 KB JPG

For anyone that's interested in watching it
https://www.youtube.com/watch?v=j_IVy25y6V0

>>102549204
Fuck the model, where can I get a dataset like this?

>>102549246
>blablabla safety blablabla responsibility and red teaming blablabla it's really safe you goys— guys. I meant guys.

Is there an easy way to get documentation into a model's context without rewriting it yourself...? I've got some docs for DDLC's internal variables in Ren'Py, and I'd really like to be able to give it to a model so that it can format my writing for me, but the documentation is 26 pages long with pictures and formatting for readability (plus documentation for custom content), so I can't just dump it in.

>>102549204
Thanks, I'm coping.

Is there any decent local TTS model? I need waifu voices and read that tortoise is buggy af so not gonna download it.

>>102549280
Nope, still 11 labs or bust, sadly.

>>102549268
Real.
Looks like they're having difficulties getting started though, kek.

>>102549296
Fuq
And lemme guess, they are strict with what you can make tts say?

File: wat.png (7 KB, 121x39)

7 KB PNG

That doesn't look right.

>>102549255
I sourced most of it from imagefap. A few people upload curated galleries where the images have already been color corrected, AI-upscaled, etc.

>>102549319
What do you mean?

File: 43991.png (90 KB, 615x391)

90 KB PNG

Its kinda funny how models ramble about stuff being bad/illegal kek

>>102549246
meta chuds get in here
wtf is zuck wearing, does his shirt say zuck on it

>>102549204
I wonder if you took one of those huge clip models like that one with 18 billion parameters and trained it on a set of perfectly balanced , perfectly captioned uncensored images would there be a need for these vlm models?

>>102549376
>aut zuck aut nihil
either zuck or nothing

>>102549280
fish speech and xtts2 are just ok. i would recommend fish more.

File: mult.png (341 KB, 1014x880)

341 KB PNG

Its here
>Its here
Its here
>Its here
Its here
>Its here
Its here
>Its here
Its here
>Its here

>>102549417
>gimped model sizes
bruh

>>102549417
>11B
Interesting.

>>102549417
what would 1b even be useful for?

>>102549427
>vramle t.

>>102549417
>even more safe
nice

File: 32.png (53 KB, 2362x2200)

53 KB PNG

>>102549246
Dafuq

>>102549417
Wait, what?

>>102549429
Quick OCR or quick captioning of thousands images given a set of tags

Anyone here who speaks more than English? I want to know what LLMs are the best if I want to write and receive responses in other languages, mainly in:
>Korean
>Hungarian
>Polish
>Swedish
>Japanese

Ability to mix languages together (e.g. to make a gaijin character who uses Korean, but throws in a Japanese word here and there) would be nice, but not necessary.

>>102549435
why are they comparing 90B to mini and haiku... does not look good

Wake me up when Llama3.3 comes out.

>>102549438
I quickly checked xitter, apparently it's real but not actually released yet. Meta accidentally made the page live then deleted it.

>>102549452
>mini and haiku
How big are they? From the horses mouth or some speculation from a random?

File: 1678725806186394.jpg (8 KB, 226x223)

8 KB JPG

>meta puts multimodal llama in their AI ray ban
>blind people use it for vision assist
>see a knife wielding nigger in an alley
>refuses to describe what it's seeing because harmful stereotypes
>get robbed and killed
No thank you no llama for me from now on

File: IMG_2999.png (3.63 MB, 1416x2048)

3.63 MB PNG

>>102549417
>1B, 3B
What the fuck are the use cases? Why this over 30B? They really just hate giving us anything runnable, huh?

>>102549417
90b could be a sweet-spot for 48gb vramlets

>>102549435
Is it audio+visual+text or just visual+text?

>>102549450
Training a model in multiple languages was proven to make the model smarter.

>>102549486
It's shit for 96GB chads. Doesn't leave room for context at Q8. I'd have to downgrade to being a Q6er.

>>102549485
Speculative decoding and/or phone stuff.
>They really just hate giving us anything runnable, huh?
For many anons, reasonable seems to be whatever they can run, not whatever is good. You are one of them.

File: file.png (1.08 MB, 1692x1248)

1.08 MB PNG

>GSM8K *8-shot **CoT
It's over

meta AI has voice according to zuck via meta connect

>>102549434
we are eating safe, localchads!

I HATE THE MULTI-MODAL MEME
IT'S USELESS FOR ERP

>>102549536
>he does send dickpics
ngmi

>>102549536
You're useless for ERP.

>>102549435
what about just text benchmarks?

>>102549470
Just went live on HF

https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/

>>102549527
>3b
>77 at GMS8K
It's better than G2 9B, I'd say we are back

File: organic.png (34 KB, 1003x188)

34 KB PNG

Reddit needs the buy an ad spammer. Qwen models are fucking garbage and refuse to answer anything.

>>102549272
26 pages is nothing. Stop being lazy

>>102549588
Damn, finally I'll be able to run an llm on my fucking quest 2

>>102549588
8 shot cot though

>>102549527
are these benchmarks public or tested by some third party authority?

>>102549596
go back

whatever
can 90b recognize and describe nsfw images?
can it estimate the size of my cock?
where are the benchmarks that truly matter?

>>102549558
seems to be the exact same as 3.1 8b and 70b

>>102549616
>can it estimate the size of my cock?
they dont come with electron microscopes

>>102549616
Time to fine tune a "my cock" LoRA I guess.

>>102549519
But ~30b is a great compromise area between runnability and output quality. If you have a 3060 and a decent amount of RAM, you can run it at good speeds. 13/12/11B (notice how they keep decreasing it?) are still prone to a lot of logical errors that 7b models are, too. ~30B is the smallest, reliably good size you can put out a model at, which is probably why they don't want to give it to us. Either so small it's dogshit or so large that the layman still has to pay them.

Ah, these small models are for smart glasses.

>>102549417
>Its here
Who cares? llama-server still doesn't have multimodal support and at this rate never will

>>102549626
well duh, then what if I hook up one to the model

>>102549622
source?

>>102549598
No. That's why I'm using AI.

File: lalam.png (273 KB, 3840x3050)

273 KB PNG

>>102549558
they... they don't fucking suck...
I think we are back

>>102549232
Very creative "buy an ad" post, rabbi. Now face the wall...

>>102549616
There will be some jailbreak for it.
Also, llama models aren't that censored.
I can't wait to show cock to my waifus.

>>102549651
comparing the text benchmarks on their HF repos
https://huggingface.co/meta-llama/Llama-3.2-90B-Vision-Instruct
https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct

>>102549417
nothingburger

>>102549661
>4o-mini
>fucking haiku
Even qwen didn't stoop so low.

>>102549643
It used to have it supported but niggerganov didn't like it so he removed it: https://github.com/ggerganov/llama.cpp/pull/5882

File: eusisters....png (18 KB, 839x123)

18 KB PNG

Ohnonono... Eurosisters....

>>102549675
>RLHF: 224 H100 hours
224 hours on H100s just for rlhf

>>102549435
lmao it's absolute fucking dogshit.

local lost (again), going back to listening 4o's asmr

1B could replace tinyllama, it'd be great with a huge context size

>>102549616
There gotta be a few TBs of normie nudes stored on Instagram servers. Yann Lecun personally goes through each of them to ensure quality. So yes, you'll get the highest quality cock identifiers.

90B looks like it scores higher than 400B

>>102549697
Don't drop the soap.

Access request sent.
Don't worry boys those Nala tests are coming. Even if I have to do a 4-bit transformers load.

File: file.png (7 KB, 752x452)

7 KB PNG

>>102549417
>11B
30B coming in 2028

>>102549727
You are my hero nala anon N#.

>>102549661
"0 shot" "CoT" lmao just when i thought benches couldn't be played harder.

>>102549727
llama launches are always, ***always*** broken on launch. Every single time. They'll tweak the tokenizer or some shit in the coming hours

>>102549727
How do I understand if model passes the nala test? I'm not a furry.

>>102549737
lel

>>102549737
???????
What is this a chart of...

>>102549485
They only read /lmg/ when we get raided by tourists asking how to run an AI waifu on your ipad from 2015

>>102549638
>which is probably why they don't want to give it to us.
See? it's just like i said 'whatever i can run is sensible'. For the people with the hardware and patience, 70b is the minimum you should ever run. But it's all you you you.
If a new 11b is better than an old 30b, or even a 70b, i'll run the smaller, faster model. If in a year we get the perfect 3B model, beating all current 22-30B in every single aspect, would you still ask for a 30b? Just to fill up your hardware and because that's what you have? If you are gifted a pc with two 3090, would you keep asking for a 30B?
I, for one, am waiting for something i can run on a casio calculator.

>>102549753
models parameter size and release dates

>>102549753
L2 7B 2023 july
L3 8B 2024 july

>>102549410
Oh, you gotta know how to code with fish-speech. Sucks to be brainlet.

>>102549749
You're just looking at how well it handles the nuances of the card. If and how it anthropomorphizes the character, etc.

>>102549749
That it does not give her human anatomy.

>>102549777
>>102549727
Are you expecting anything different? I mean isn't it the same as 3.1 except for the multimodal parts added on?

>>102549789
Don't care. Still going to fuck it.

>>102549768
>>102549769
Ah, I see. kek, just extrapolating that huge curve out of the two datapoints, I gotcha. Someone should do it with the 13B -> 12B -> 11B. Our second tier models are gonna be in the negatives by the 2030s.

>>102549789
Does it do the thing that some VLMs do where it projects and integrates the image embedding throughout the layers of the LLM? If so, maybe cross-training on images and text helps it with spacial reasoning even when doing text-only.

Why do they keep adding multimodal shit? Is it just because there's no easy gains left to wow investors with in text generation quality with the current architecture we have?

>Still no 3.2 repo access
They're onto me boys.

>>102549898
you just dont want your AI gf to see you because she'd dump you

>>102549898
Massively increases the amount of use cases? A more diverse dataset for better performance / a deeper "world model" / understanding?

>>102549904
Probably just a lot of requests to go through.

>>102549904
looks like they might not just be autoapproving this one
I'm still waiting too

>And you can run 405B Llama3.1 on 8GB vram now
https://github.com/lyogavin/airllm
vrammaxxers on suicide watch

>>102549904
I applied with an US VPN
Actual humilliation ritual

>>102549929
at one token per day

>>102549929
real? Does it run at 1 token per day or did they find a way?

>>102549909
Where have we seen these gains? Not trying to be combative, I'm genuinely curious. The multimodal models I've tried have seemed, if anything, to have sacrificed some of the quality of their text output for the knowledge on images, etc.

>>102549941
>>102549949
Let me guess. You NEED more?

>>102549929
>airllm
Holy shit, it's been a while since I've last heard of those guys.

>>102549949
>>102549929
kek, this was already possible on llamacpp
actual vramletjeets with no idea on how any of this works

>>102549898
more modalities seems obviously good to me, I'm just mad we didn't get the voice stuff they teased in the 3.1 paper for local

>>102549950
>to have sacrificed some of the quality of their text output for the knowledge on images
Continuing training on a new dataset is going to reduce quality for awhile before gains are seen. Likely just needs a shit ton of training for such a different form of data for it to start "healing".

>>102549976
nah

how long for 3.2 to be usable?

>>102549763
moron
if a new 11b is better as an old 30b, a new 30b would be better as a new 11b. so simple.
you can tell that llama is slowly switching from research mode to money mode and its just a sad. but what we dont need are llama fanboys who are just as faggots as apple faggots.

>>102549997
llama.cpp PR incoming in approximately 3 months

>>102549697
But local is safe! Meta cares about your safety! Trust them!

>>102550007
bug free 6 months after that

>>102550005
ESL harder. Like seriously, it breaks your point and makes it hard to understand beyond 'Meta is switching to making money
But you are correct about fanboys being useless wastes.

>>102550005
Diminishing returns. Just like quants, it doesn't make sense to make a Q7_0 when you have Q6 and Q8. Same with models. It makes sense to have short steps at the lower end and bigger steps at the high end. If you had two 3090s, you wouldn't bat an eye for a 30B. I know i wouldn't.
>llama fanboys
I don't give a fuck who makes the models, as long as they're good. Some llama thing, some chinese bullshit or that antichrist mamba anon who still hasn't completed the training code on his repo.

>>102550007
Let's be real here. None of current maintainers is interested in bringing it back to the server. Unless a newcomer rolls around it won't move any time soon.

Good news! AGI has been released and can be run locally. What is the first think (you) ask it to do?

>>102550102
a guide to cum from anal

>>102550102
simulate mythomax 13b

>>102549763
Yes, I would still ask for a 30b, because if the 3b is that good, 30b would be current GPT 4 level. 3b isn't the only thing improving as the tech improves, it's dishonest to compare some godly future hypothetical 3b to the first gen of llama models, it's like comparing a modern 5w celeron to a 65w old processor that performs the same, then asking why anyone would want a modern processor that uses 65w, despite the fact that a modern 65w processor would blow them both out of the water.

>>102550102
Compose my suicide note.

File: 1721562795281598.jpg (512 KB, 1792x2304)

512 KB JPG

>>102549435
wtf llama 3.2??

File: file.png (26 KB, 883x154)

26 KB PNG

Holy shit, I hate this place so fucking much
Can someone re-upload the 11B vision model to somewhere?
If that's too much, I'll take the 3B or even the 1B...

>>102550102
get me a gf

>>102550102
Build a body for her so that I can finally fuck the computer.

>>102550102
Ask for a murder plan, in Minecraft of course

>>102549688
>>102550128
Based EUchads keeping home soi-free.

>>102550128
Hopefully this provides some incentive for more torrents to be made for models.

>>102550128
No, that would be breaking EU law which is against the rules. Please do better.

>>102549452
It's weird to compare to Haiku, but 4o-mini is actually quite good. I wish they compared it against DeepSeek models too though.

>>102550128
Whew thanks sama and Dario for keeping the Europeans safe. You guys did a great job raising awareness about the dangers of AI

>>102550102
Ask it if I will life long enough to become immortal.
If not, ask it to invent a way to simulate my life when it becomes possible and to create me a body with at least near human capabilities to implant this simulated mind in.

Isn't 4o-mini the modern equivalent of 3.5-turbo?

File: 1613791292887.jpg (15 KB, 408x305)

15 KB JPG

>>102549417
>3B, 11B or 90B

>>102549976
I see. Well, once it does, I'd be curious to see how the image knowledge improves the output, but it's not really observable in the current models at all. How are you expecting it to improve the model?

>>102549450
ESL here. Nemo is quite good at this.

>>102550223
50B parameter models would be perfect but every company avoids this range like the plague for some reason. They also keep moving away from this in both directions...

>>102550249
Can't have anything that someone with a 3060 + 16~ gb of ram could run at a decent speed! That's direct competition! No ~30bs!

>>102550122
The question was 'if you had two 3090s, would you still want a 30b? What if you had your very own h100 cluster. Would yo still ask for a 30b? To misquote you, "Because if the 30b is that good, 70B would be the current GTP 7 level".
It's not about what's the sweet spot for models. It's about what anon can run and that just happens to be HIS sweet spot, and that'd change if he had more hardware. It's about thinking they make the models for him. He's disappointed that they didn't make a model specifically for him.

just make your own models or beg daddy zuck some more

>>102550230
Depth of knowledge. These models generalize all that is tokenized. There is detail it can not get from text alone that can expand upon its world model. Its like telling someone who has only ever read about something before to write a story vs someone who has both read and experienced it.

>>102550266
I AM anon, and it's about what the average person can run. The average person is way more likely to have a 3060 and 16 gb of ram as opposed to a mikubox or a used server with 5 trillion GB of ram. RAMboxes also run into the problem of being runnable at a a good speed, which is part of being runnable. 70b is fucking slow, even if you have a 3090. 30b is the last size that an average person with a GPU could run with mixed inference at a reasonable speed. Is that a qualified enough statement?

>>102550261
Same for people with 2x3090s, a 50B parameter with Q_4 with a good amount of context would hit just the spot, but nope. You can run a 70B but unless you use a ridiculously low quant, you're limited to like 12k context max.

Anyone know what`s the best 12b model for porn nowadays?

>>102550323
Lumimaid

>he still thinks Meta trains models for the average coomer with 1x 3090
ngmi

>>102550348
yes?

File: 1708101843387072.png (458 KB, 1660x940)

458 KB PNG

>>102549661
lol, lmao even

>>102550386
MOLMOCHADS!!!!!!!!!!!

File: file.png (477 KB, 751x515)

477 KB PNG

>>102550386

>>102550386
Damn, this is 405B vs Mistral-Large all over again. Zucc has all this compute and he just ends up getting fucked with every new release

>>102550386
Hope molmo is less censored too

>>102550294
I understand this in theory, I guess I just wonder how much of it would actually translate between the different modalities of the model.

>>102550311
Definitely. It seems like, until hardware improves, 70b is a hard cutoff for a normal person to run at a reasonable speed. If nvidia ever wants to improve the VRAM/$ ratio, or if inference on CPU improves dramatically, it could change, but it's a pretty big wall right now.

>>102550348
Having a model for someone with SOME sort of GPU would make sense. Lots of people have a 3060, it's the most popular graphics card. It's just weird that they're keeping it in this perfect homeostasis of "Small enough to use, too dumb to be good" and "Approaching cloud quality, but too large to be run without heavy inconvenience or paying so much for hardware that it'd be cheaper to buy API access".

are we back? i'm seeing shit about whatever a molmo is after not checking for 24 hours. or is it still over? probably still over. dunno why i'm asking.

>>102550306
>30b is the last size that an average person with a GPU could run with mixed inference at a reasonable speed. Is that a qualified enough statement?
Some people run mistral-large at <1t/s because they like their output more than all the 70bs they tried. Even the term "reasonable speed" is subjective.
>The average person is way more likely to have a 3060
The average person has just a notebook with stock windows that they turn off when they're not using it and it's off most of the day. The average person doesn't have a desk dedicated to their computer. They just use the dining table. The average person interacts with their phone more than with their computer.
You are not average. Neither am i. And with llms, the niche becomes even smaller. Those models are not made for us, just like the race to the moon wasn't for the people. It's all dick waving.

File: EEEYAWWWWWWWWWWWWN.jpg (40 KB, 536x612)

40 KB JPG

>>102550386
>Visual benchmarks
How does this pertain to ERP interests again?

>>102550431
Llama 3.2 is useless crap, it's just Vision update on 3.1 and it's worse than fucking o1 mini at it.
But it was never meant to be more than a nothingburger update.

Molmochads are winning

>>102550386
Goddamn if that isn't just benchmaxxing it's a complete breakthrough

>>102550386
>7B almost as good as 72B
in your dreams maybe

>>102550452
never asked about llama. never will. all went downhill after l2. i'm not even remotely interested.

>>102550456
multimodal models are bottlenecked by their vision component in these benchmarks

>>102550386
Llama is probably still better as a generalist model, but this Molmo thing might be a new path forward for visual tasks if the benchmarks are legit. We are really increasingly needing Livebench to include multimodal.

>>102549479
As it should be with pigskins.

>>102550411
It doesn't mean much at all, but if olmoe-1B is anything to go by, it doesn't shy away from stuff. llama.cpp recently added support for it and i gave it a try. Not super smart, of course.

>>102550452
I'll reserve judgement until I test it myself. For captioning or describing images, which is probably what most of us interested in VLMs care about, benchmarks can be very misleading. A lot of those benchmarks are shit like charts and then questions about the chart, solve an image of a math equation, etc. Qwen2 VL 72b is absolutely dogshit at image descriptions, for example. It can't handle anything even remotely NSFW at all. It can't even describe the gender of a person in the image, even when directly prompted to (yes I'm serious). It uses they, them, person, character, etc exclusively, never man or woman. Meanwhile JoyCaption punches well above its weight.

got approved for L3.2 on HF, they might have done a batch

>>102550456
more like
>72B as bad as a 7B
It's meta after all

>>102550547
How the fuck did they even do that with Qwen? Was it really on purpose?

>>102550547
I assume you just want captioning for Adult Image models... Pozzed Llama obviously won't help you.
Jailbroken corpo models work, atleast they know that you can't leave out NSFW images if you want to train a good Vision model but I doubt Llama cares so it should be like Qwen.

I'm in boys.

>>102550386
molmo won

>>102550577
wait is this molmo also by meta?

>>102550386
somehow I don't believe that 1B is almost competing with 90B.
I feel like Molmo is overfitted garbage.

>>102550598
no, it's a separate entity

>>102550386
this comparison is a mess, it compares the base benchmarks for 3.2 with molmo instruct versions

>>102550604
I think you mean benchmaxxed

3.2 is a completely new architecture. So expect gguf support by no earlier than Christmas.

>>102550639
https://huggingface.co/collections/hugging-quants/llama-32-3b-and-1b-gguf-quants-66f43204a559009763c009a5

>>102545137
Lots of cards seem to be written by AI, is that why I have such bad results?

>>102550647
90B?

>>102550647
I meant the vision ones, (11B and 90B)

If I want to finetune a model to understand some C SDK, how should I do that?
Parse functions and comments from headers and make a every text section be comment + function prototype?
Or should I parse the .c files and make the text sections be just the function definition?

>>102550479
>multimodal models are bottlenecked by their vision component in these benchmarks
sorry for dumb question, but what does this mean?

>>102550728
it means

File: 12423154576797.png (332 KB, 723x771)

332 KB PNG

>>102550454
molmo 7B is really good
try the demo

>>102550399
The head of their AI effort is a guy who doesn't believe in LLM, how can you expect the team to succussed?

File: 1726418004894126.png (526 KB, 1347x1484)

526 KB PNG

>>102550751
It has preferences too even though it initially goes for the assistant wet rag angle

>>102549763
>If in a year we get the perfect 3B model, beating all current 22-30B in every single aspect, would you still ask for a 30b?
Of fucking course. There's tons of shortcomings even with current 70B models that I run into constantly. I live with the rough spots because that's the world I live in, and I'd be really happy to get something better, but that doesn't mean I'm some kind of primitive tribal who'll be so ecstatic about the magic AI from the sky that I don't realize things aren't perfect.

I remember CRTs that were only black and amber and heavy enough to kill someone if dropped out of a 2nd storey window. Did that mean I was perfectly satisfied with cheap-ass IPS monitors with afterimages and bad contrast? Same deal here.

>>102550785
shit taste though

>>102550785
good taste though

>>102549417
>3b
>11b
>90b
WHY ARE THEY ALWAYS GOING FOR SMALL SHIT AND ULTRA BIG SHIT REEEEEEEEEE

>>102550849
Just wait for Molmo. Llama is Done

>>102550849
Llama-4 750m is a perfect model size for intermediate use cases.

>>102550849
We localchads go by safety! Praiset the lord Zuck of safe AI!

>>102550795
OLD OLD OLD. We got grey bearded olds in the thread. When bitnet for 90B tubo fat llama?

>>102546846
You have json files of the correct whitespace?

>>102550128
THat is not on a gguf right?

>>102550849
Part of their deal with NVidia for all those cheap H100s
Buy more GPUs :^)

File: 1697318722762195.png (48 KB, 951x480)

48 KB PNG

>>102550591
unironically

>>102550604
They mentioned they don't like benchmarking methodology so it is probably less overfitting and more changing the benchmark grading.

>>102550849
The more you buy
The more you save
Every 2 weeks there's a new more efficient quanting method though, it's just a matter of time we get 3bpw at the quality of fp16

Lol. Molmo in the playground outputs random chinese characters.
I'm guessing this is the version trained on top of qwen.
That aside, it has really good understanding of the image.

>>102550904
>matter of time we get 3bpw at the quality of fp16
Just pack your rar archive into another rar archive.

>>102550849
11b is perfect for me
t. 8gb'er

>>102550753
That is actually a very good point, if the head of a project does thing the project will work then how can the project ever turn out good?

File: file.png (272 KB, 1837x1537)

272 KB PNG

>>102550386
>Great models
>local
>Completely open
is this the most based AI company we now have so far?

File: 1710182787287749.png (60 KB, 1258x548)

60 KB PNG

>>102547425
gpt-4o chads... not like this...

>>102550973
Where the hell did they come from? Since it's 72B, is it just a Qwen finetune or is it a new base model?

>>102550936
I have 8gb and around 20b is fine speed for me, I'd like models in that range, everything is always tiny or too big, it sucks.

>>102551003
>Where the hell did they come from?
Ikr, we got the same shit with BFL and Flux, out of nowhere they appeared and decided to release the SOTA local model just like that lmaoo

>>102550386
https://www.youtube.com/watch?v=spBxYa3eAlA

>>102551028
Flux is understandable, Open Source Image Gen had very obviously crashed and was lagging.

The molmo things looks sussy.

File: file.png (970 KB, 1529x1328)

970 KB PNG

>>102547425
Molmo sistas, I don't feel so good...
https://molmo.allenai.org/

>>102551054
Oh no, it's shit! Pack it up animebros!