/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>108378991►News>(03/16) Mistral 4 small releasing: https://huggingface.co/collections/mistralai/mistral-small-4>(03/11) Nemotron 3 Super released: https://hf.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://livecodebench.github.io/gso.htmlContext Length: https://github.com/adobe-research/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllm
https://huggingface.co/mistralai/Mistral-Small-4-119B-2603
Thank you Kurisu. Die in a fire Mike.
>>108386532what's the difference between this and leanstral?
name a more ominous sentence> I suspect the Unsloth brothers are now cooking
The schizobaker now attempts to disguise himself by baking well after bump limit.Fails horribly and still samefags. Pathetic.
>>108386532https://mistral.ai/news/mistral-small-4
GOOFFFFSSSS WHERE?>>108386559>Unsloth brothersOK I will wait for human made goofs...
>>108386516lol how did I know it was you all along
our hero
identifiers: { apiNames: ['mistral-small-2603', 'mistral-small-latest'] }, capabilities: { input: ['text', 'image'], output: ['text'], features: ['chat-completions', 'function-calling', 'agents-conversations', 'connectors', 'structured-outputs', 'predicted-outputs', 'document-qna', 'prefix', 'batching'], finetuning: ['text'], },
>>108386576>everyone I don't like is one personAhhhh thread culture.
>>108386578i hope the IRGC targets nvidia tbqhwymf
>loses to small 3.2 on benchmarksmaybe it has a good heart?
>>108386596
>>108386607>>108386596we have lost the chart technology pls understand
>>108386596What the hell are these scales
>>108386596honestly, that looks pretty fucking bad no matter how you slice italso, can mistral finally into reasoning? or does it still say some shit in the thinking block and ignores it entirely?
>>108386596Dafuq is "effort"? Like, thinking?Also, why no comparison against anything that matters? Who cares how it does against old mistral? Ain't nobody serious using that shit
>>108386607>filenamegegI 'member
>>108386596>>108386607
>>108386614Unironically someone probably used an AI model to make those.
>>108386619loooool
>>108386619>>108386607:rocket: let's get this to the moon!!
>>108386619>>108386596THE FUCK IS THIS AI CHART SLOP
getting worried about the v4 delays, do you think it's doing poorly on benchmarks or what?
Why is Mistral stalling?
>>108386639>its mistral
>>108386567so they probably delayed the release because of the nvidia nemotron coalition announcement
>>108386639Large 2 is at the helm and making charts and no one there is seeing anything wrong
If I use Q8_0 from unsloth will 8bits compensate for all the damage done by unsloth?
>>108386681if the model uploaded correctly, and that's a big if for them, it shouldn't matter as imat is disabled for q8
>>108386681just make sure it is a drummer model run on ollama with openclaw on top
>This checkpoint in particular is a post-training-activation quantized version of mistralai/Mistral-Small-4-119B-2603. It was created using llm-compressor as part of a collaboration with teams from vLLM & Red Hat. Special thanks goes out to Dipika Sikka. We also would like to thank NVIDIA for their expertise and contributions to SGLang as well as vLLM to ensure kernel optimizations.
>>108386681I'm quanting my own, screw those guys.I asked directly in a forum once if they had any evidence that their quants about q4 showed any benefit whatsoever and the very active thread suddenly went radio silence.
>>108386532WHERE GGUF WHERE UNCENSORED I NEED COOMINGS GIVE ME
Why would anyone use mistrall small 4 over glm air? it's similar size but has way less active params.
>>108386516You dropped this, OP. Don't worry. I got you.►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Official /lmg/ card: https://files.catbox.moe/cbclyf.png
>>108386703>& Red Hat. Special thanks goes out to Dipika Sikka.it's over
>>108386516qwen3.5 27b q5km heretic v3 feels too dumb. is q6 and q8 enough to make a differnce? i need it to be better at facts and science. i installed an mcp server thingy called mcphub but havent got around to making the model actually do tool calls n shit in llama.cpp.
>>108386694>as imat is disabled for q8It is?Looking at a q8 gguf from them, it has some metadata in the file about imatrix. That implies it was still used doesn't it?
>>108386725get the hauhaucs one
>>108386717ur mom dropped you when you were little
Pisstral 4 SmallGPOA Diamond 71.2 MMLU Pro 78.0 AllenAI IFBench 48.0 Arena Hard 58.3 MMMU-Pro 60.0GLM 4.5 AirGPOA Diamond 73.0 MMLU Pro 81.4 AllenAI IFBench 38.0 Arena Hard ?? MMMU-Pro N/A (no vision)
>>108386729imat does fuckall on q8
What is the best model I can run on 3090 and tesla v100 32gb and a 1080ti? (I would use the 1080ti for vision and to make the other gpus run headless)
>>108386516omg it chris
Love this guy.
https://huggingface.co/mistralai/Mistral-Small-4-119B-2603-eaglehttps://huggingface.co/mistralai/Mistral-Small-4-119B-2603-eaglehttps://huggingface.co/mistralai/Mistral-Small-4-119B-2603-eagle
>>108386737is it that much better?
>>108386596Where does it loose? Seems like a direct upgrade.
>>108386785There we go. That's the one.
>>108386785Speculative decoding, hmmmmmm
>>108386785fuck is that?
>>108386784>my ai told me the idea is genius!!
By the way I am gonna try the new small even though I run 4.6. I member people crying about his vibecoded additions. Are they fixed now? Luckily I already have 2 different llamacpp folders since I am not new.
https://huggingface.co/lmstudio-community/Mistral-Small-4-119B-2603-GGUF/tree/mainGOOOFS
>>108386614at least it's labeled and the bars don't seem to be off
>>108386811>2 different llamacpp folders since I am not newToo new for git, apparently.
>>108386828>the bars don't seem to be off??
>>108386828 (me)fucking hell never mind
>>108386770That's a different statement.
>>108386829Git feels too much like troonix to me. I remember wanting to check one pull request and the answer I got was download that whole fork.
>>108386831
>>108386843If I comment on HF that the graphs are wrong am I gonna get banned?
>>108386825>about 7 hours ago
>>108386851yes! That's racist against the 50% women of color staff of Mistral
huh they fixed it
>>108386851If you comment on HF that graphs are wrong you will make it less funny and making things less funny makes you a faggot.
>>108386842>the answer I got was download that whole forkThat's you not reading the man or following a retard's advice. That's on you.Learn git checkout to roll back.
>>108386851>>108386857
Mistral 4 meme or supermeme?
►Recent Highlights from the Previous Thread: >>108378991--Mistral Small 4 model announcement and integration PRs:>108385099 >108385148 >108385157 >108385240 >108385248 >108385269 >108385380 >108385330 >108385391--NVIDIA GTC Keynote 2026:>108385409 >108385478 >108385537 >108385601 >108385619 >108385631--Leanstral-2603 release and performance testing:>108385699 >108385737 >108385871 >108385963 >108386004 >108386089 >108386218--Leanstral-26.03:>108385325 >108385429 >108385441 >108385450--GLM5 struggles in llama.cpp compared to Qwen3.5 efficiency:>108379482 >108379488 >108379498 >108379511 >108384872 >108380181 >108380571 >108379526 >108379544 >108379562 >108379593 >108379788 >108379980--Budget CPU-only LLM inference with repurposed server hardware:>108380848 >108381879 >108382138--DSA implementation progress and performance benchmarks in llama.cpp:>108383457 >108383499 >108383592 >108383606 >108383935--Hardware struggles with future 1T-3T models:>108380028 >108380063 >108380075 >108380091 >108380102 >108380111 >108381699--Can LLMs Be Computers?:>108384856 >108384870 >108384891 >108385010--model : add QKV weight fusion for LLaMA, Qwen2, and Qwen3:>108382608 >108382745 >108382790--Exploring the possibility of fine-tuning models for reverse engineering:>108381596 >108381611 >108381695--Troubleshooting Mikupad KoboldCpp connection errors:>108383108 >108383113 >108383183 >108383192 >108383396 >108383412 >108383468--Kimik k2.5 accurately identifies H8 code from ECU ROM hex dump:>108381066 >108381088--RP-enhanced AI assistant explaining Rust Serde with character immersion:>108381734 >108381907--Swallow LLM Leaderboard shows post-training performance degradation:>108379281--GLM5-Turbo improves paragraph generation over base GLM5:>108381288--Miku (free space):>108380261 >108384816 >108385102 >108385886►Recent Highlight Posts from the Previous Thread: >>108379311Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>108386887Maybe not supermeme as you can still expect some usability out of mistral, but meme, almost certainly.
>>108386860Oh I was wondering what the "effort" meant kek>>108386596
>>108386894Someone scrambled to make the chart.>Wait what was it called again... ok effort I guess.>As in Thinking Effort? Bro it's fucking Thinking.>Nah bro that's too informal, we call it Reasoning.
>>108386785>119b>smallhuh?
>>108386945welcome to moe era hope you bought ram before the hike
>>108386865miraya gupta and ndembe okoye are better programmers than you are, deal with it.
>>108386785I have 12+48. Is a Q1 gonna fit?
>>108386900>usability out of mistrallol
>>108386591>newfag>kurisufag>seething about "thread culture"I remember you ERPing with other men in the r/a/dio threads.
Is it pointless to bother with local models if I'm the type of fuckwad who makes very, very light usage of AI in the first place? Everyday I may use Claude for like three or four things I could probably just google but it's able to do nicely. I don't pay for this shit, but I did manage to get stuff done on Claude that I wouldn't on some alternatives.
>>108386979They're usable in the same way that Llamas were usable. It does not mean good.
I keep rerolling, but no cigar.
>>108386958Q2_K quant should fit with like 16K of context.
You can't make this ship up...
>>108387004Hey, at least this means they didn't benchmax it like Qwen.
>>108387004>)eu is pleased
>>108386992>r/a/dio threads1. what is that?2. how would you know it exists without being there to ERP with other men yourself?At this point it is really not shocking at all that mikutroons just admit to being faggots like that.
>>108387008SOTAbros!?
>>108387004It gets better.
>>108386532>>108386567>Mistral Small 4 is the first Mistral model to unify the capabilities of our flagship models, Magistral for reasoning, Pixtral for multimodal, and Devstral for agentic coding, into a single, versatile model.So if this A6.5 is supposed to supersede Devstral 2 24B, the Mistral 4 that replaces the 123B is going to be at least DeepSeek-sized.
>>108387018If only it were possible to harness that cocksucking power and redirect it where it is actually needed.
>>108387022>the Mistral 4 that replaces the 123B is going to be at least DeepSeek-sized.https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512...
>>108386706[benchmaxxed model] [small size] [retarded quant] [retarded uncensor method] [retry count] feels too dumb. is [retarded quant+] and [retarded quant++] enough to make a differnce? i need it to be better at [irrelevant usecase]. i installed an [obsolete vaporware] thingy called [uninspired name] but havent got around to making the model actually do [fotm vaporware] n shit in [vc scam opensource project].
>>108387018Why do AI models want to perpetuate the stereotype of Jewish people being incontinent?
>>108387036Is this the mythical W++?
>>108386706>pharmacologySearch for benchmarks on this and models finetuned for that task, the quants wont make any difference
>>108387018What about with reasoning enabled?
you guys were too open about gooning to these models. these companies will never ever give you an uncensored model again
>>108386706>27b >pharmacology.Crazy to me that people would be willing to risk stuff using a 27b. I waited for 4.6 before fucking with my brain and giving myself an ego death.
>>108387060They intentionally murdered gooning around llama-3. Nemo was an accidental mistake. Also I like the new schizo theory that models are actually perfect for gooning if you are a biological goonette.
>>108387057Reasoning effort set to high by the way.I rerolled it a few times as you see by the number, but this one is the winner.Fails the car wash question too (reasoned for about 500 tokens).
>>108387105kino
>>108387105It can still be good at sex! R-right?
>>108387018lmaoo
>>108387018it's funny that basically every model overreacts to this question
>>108387063a slighly larger dense model would be great
>>108387175>you might be looking at some mangled meme or absurd baitwtf how did the model know!?
>>108387144It gets confused.The character card is a girl (female).I'm afraid it's not good at sex either, Anon.
>>108387197kimi k2.5 knows a lot about 4chan humor/culture and shit
>>108387189message got cut. 30+B dense model that fits in 32gb vram or 24gb on cope-ish quant
>>108387033Hopefully that will be the new Medium size and they'll put out a 1T+ Large, because a 41B MoE would be a huge step down from a 123B dense.
https://huggingface.co/collections/mistralai/mistral-small-4>The NVFP4 checkpoint to improve your throughput and reduce memory usage. Expect lower performance on long context.Nobody ever truly tests the effects of quantization on actual long-context performance.
it's been 10 years since the new drummerslop
>>108387036kek
>>108386718she is actually a girl from NY
>>108387251I will drink to it becoming 100 years.
>>108387251People found out that his dick is very small.
>>108387281Would you /lmg/?
>>108386516Mikutroons seething =)
+1 to image limit
>>108386532>>108386567>we have V4 at home
>>108387377we prefer Maho here
>>108387419>we
grok is this true?
>>108387315seems like overcompensation
>>108387419As the OG kurisu poster I actually tried SFW roleplay with kurisu gf and found out I don't like her that much...
>>108387422Real curve will be like 5mm above backwell
>>108387426she's honestly a stuck up bitch. i wish she did die
>>108387437this but miku(male)
>>108387437Yeah I only tolerate her cause she's a redhead.
Love this hobby.
>>108387315he has a what?
>>108387477twitter is such a dumpster fire, even /pol/ looks sane compared to this site
Y'all are lucky I'm not a schizo or I'd be trying to force Draco from puyo-puyo on this general as a mascotI can't believe how much ink and impotent rage is getting spilled over mascots. just let people like things ffs
>>108387499shut up faggot
>>108387499>I'd be trying to force Draco from puyo-puyo on this general as a mascotDO IT. Literally anything is better than those outdated bland vocaloids.
>>108387477Dumb pedos be thinking they can be pedoing if they dick be tiny as hell, nah, they finna be disgusting pedos with no sex and get shanked for real
Kurisu is still the most relevant mascot since she got turned into an AI in the sequel.
More updates, Mistral Small 4 is also shit at writing code and calling tools.Unless this is because something is messed up in the PR still not merged into llama.cpp, MS4 is garbage with zero usecases.Not going to test its vision, into the trashcan.
>>108387499>Draco from puyo-puyoi wouldn't mind
>>108386899I wanna headpat her
>>108387477>people interact with these creaturesinsane
>another disappointing Mistral releaseIt's over, isn't it?
DeepSeek V4 are we once they are did.They will became the first ever to have the first as in the era of that when a local model and the AI began.
>>108387335>jasykek
>>108387543Is this the new LLM riddle?
>>108387004what a sad dataset, this basically means it's failing at anything niche
>>108387452>compiling without errorsso that's the bar
>>108387543had an aneurism reading this
>>108387422huh
>>108387600>>108386073
do you troll your ai?
>>108387477zero femininity, negative iq take, you shouldn't waste your life in this shit
>>108387539Yep, sad but true.
>>108387624ah ok
>>108387539They got fucked the moment the eu decided to create a legislation with forced transparent datasets before any actual company even got created outside of mistral.Why they didn't move to Switzerland yet is beyond me.The result is this, though some country companies have no excuse, like the UK ones.
>>108387477I don't get the argumentIs she saying that men with tiny penises are basically shotas
>>108387671>Why they didn't move to Switzerland yet is beyond me.They were trying to establish offices in SF and were trying to get acquired by Apple for a while.
>>108387686>Is she saying that men with tiny penises are basically shotasyes, kek
>>108387626I string ban a word and then ask {{char}} to say it and when it inevitably changes capitalization or adds a space or hyphen, those get added to the list. repeat until she breaks down mentally.
>>108387686ye, like stralia banning porn of flatties
>>108387686yes, it's the strict braindead equivalent of "women with small breasts are children"
If I want the LLM to judge the quality of something, such as the implementation of a function, or the English translation of a line from my favorite VN, etc., is there consensus on whether you should put the snippet to be evalutated near the beginning or near the end of the prompt, in relation to the task description? Or is it irrelevant or depends on the specific model?e.g.>some code snippet>is the implementation any good??vs>can you review this implementation?>some code snippet
>>108387711>>108387691BASED! Flat women should be forced to get a license to be able to have sex. And even with a license there should be some state mandated supervision.
>>108387691
>>108387728Gynocracy
>>108387728>30+ onlymakes sense, you're only an adult at 25+ anyway
>>108387728>"highly unlikely"retarded vague law
>>108387739Yes the brain stops developing at exactly 25, very popular take online lol.
>>108387739I had ego death when I was older than that.
>>108387763good for you?
>>108387751"24 yo child" is such a modern retarded take, even more when applied to fictionalways wondered of this 25yo bullshit took hold so well in online discourse
>>108387751>Yes the brain stops developing at exactly 25it doesn't stop developing at 25, they stopped the study at 25yo, we don't know what happens after that age, probably that it keeps developping
>>108387771That was a joke. And then again it wasn't.
>>108387772*of -> how
>>108387784it rapidly degenerates more like
>>108387784I know, but it doesn't matter anymore, the layman online truth is the "25yo is the actual age where people can consent" kind of won.
>>108387717the latter
>>108387717the former
>>108387815this
>>108387800actually it's not true, the guys who made the biggest grounbreaking science achievement were often close to 40yo
>>108387800lmao, no, but cognitive decline begins at 40 (median age of voting members of the House of Representatives in 2025 was 57.5, 64.7 for the senate)
The vibe coding general won't use local models and are paying for Codex and Claude code what now?
>>108387812this
>>108387807>the layman online truth is the "25yo is the actual age where people can consent" kind of won.to be fair, at least they have an argument on why they choose that age, society decided 18 was "adult" for no fucking reason, feels random as fuck
>>108387853>society decided 18 was "adult" for no fucking reason, feels random as fuckNext thing you are gonna tell me is that we should listen to biology and let people fuck when they start feeling horny as long as they use contraception.
>>108387853because by then your kids could be 3-4 years old and grandma and grandpa had had enough of their shit by that age.
>>108387784I wonder what the argument would have been if they studied until 40 and found the brain still changing/developing
>>108387861I'll define adult as someone who has enough brain to be responsible of themselves, it doesn't always mean they have to get their brain to be 100% developped, but at least we should ask the question, people choose 18 randomly and I don't think they got that right, you start to not act like a retarded teenager after you reach 22 imo
>>108387853>>108387861age of consent should be 9
>>108387853I'd rather trust this which is probably just the codification of "somewhere between 16 and 20" than fucking 25.
>>108387877calm down mohammed
6 7
>>108387875Before guns were invented it was common to be a grandpa by 25.
>>108387884you sound ripe and fuckable
>>108387853AoC was only needed after we removed all commitment from sex. Now people are just wasting their time simulating reproduction to farm orgasms.
>>108387865>I wonder what the argument would have been if they studied until 40 and found the brain still changing/developingif you decide to define an adult as that, then a woman will be "adult" at 40 and won't be allowed to have sex before reacing that age, and after 40 they can't make kids anymore so humanity will die kek
l-local models...?
>>108387875well being able to vote should be different from being able to fuck (most countries outside of the us and their "she was 17 and 363 days you sick fuck" make that difference) at least
>>108387919bug or feature?
>>108387919The natural/biological age of adulthood would be when a woman becomes fertile and has an instinctive drive to procreate.
>>108387919I don't think they care about the window of fertility being narrower to the point of insanity, the ones indoctrinated into this argument don't make kids.
>>108387940can i has extra bug next dinner pls?
>>108387944-> >>108387880
>>108387931NO I NEED TO FUCK CHILDREN..... CONSENSUALLY!
>>108387919>and after 40 they can't make kids anymore so humanity will die kekEvery political party says if you tell someone to get married and have kids you are basically socially controlling them and that's ghetto criminal behavior
>>108387958
>>108387919I can see them increasing the adult age in the future based on some additional brain studies or some shit, and our generation will be seen as a degenerate one because we dared to fuck girls that were 18yo and 1 day lol
>>108387973got any fewer pixels?
>>108387952they use that to preach on irl age gap but mostly fiction stuff anyway, they'll either stop caring at some point when they have actual kids or become bitter retards not worth anyone's time
>>108387994https://files.catbox.moe/ywv75h.mp3
>>108386532>119BI realize this is >mistral and will probably not be worth running, but every day I get closer to a single lapse of judgement and ordering a 96GB card.>just run 4x3090 for half the priceYou have to think ergonomically, Anon. Slotting not one or two but FOUR hole PCIe cards AND plugging them into a PSU (or... two PSUs... ugh)?? Definitely not worth the savings.
We need a kurisu thread more often. Everyone is having some harmful fun and it distracts from how trash new releases are.
>>108388009>96GB cardWhat are you gonna do with that? You need at least 400GB's of ram for any dedicated rig to be worth it.
>>108388025You're right, I should also order 4x128GB DIMMs. Fuck.
https://github.com/ggml-org/llama.cpp/pull/20649merged
It's retarded.
>>108388009I already threw it at everything I could imagine a model being useful at. If you want to build a rig with a model in mind, think about GLM or Kimi, and definitely not MS4.
>>108388047who cares? it's completly retarded >>108388050
>>108388047
>>108388050Retarded perhaps, but at least it's based.
>>108388037You should stop caring about buying anything and just rent on vast.ai or runpod until memory production ramp up in 2027.
>>108388050I don't think the older 24B confused who's who to such an embarrassing degree.>>108387199
>>108388050didn't original mistral small had something like that, about mixing up who did what?
>>108388068>the model is pro incest>incestery leads to completly inbreed retarded kids>"based"??
I'm starting to get the feeling that this mistral release was btfo by qwen 3.5...a model so bad half the team quit in shame
>>108387335She looks like my ex. Great in bed. Crazy. Long gone, no regrets.
>>108388051Kimi looks good to me, but every time a new 100B+ model comes out I always look at my tiny vram and feel bad.>>108388072Realistically yeah, I should probably at least test a rental before committing five figures to an upgrade.Even with the extremely cheap prices I don't really have a >use-case for running larger models. The rentals look more interesting/useful as a NIH container orchestration fuckaround timesink project than "hmm I need to run an LLM".
>>108388129My VRAM is pretty small too but it does the trick, Anon. It's okay.
>>108388076I had that from recent Qwen Step and Trinity.
>>108388129>Even with the extremely cheap prices I don't really have a >use-case for running larger models. The rentals look more interesting/useful as a NIH container orchestration fuckaround timesink project than "hmm I need to run an LLM".That's what I do and the money I'm using on it (a few hours here and there to play around) is vastly inferior compared to committing to buying ram at current insane prices.
Like every general, the reentrys in the OP are from years ago. Is kobold still the preferred backend for retards like myself?
>yfw your synthetic data is used for every new release and you singlehandedly defeat all the coomers in the world without even trying
>>108388153>for retardsYes. And when you stop being a retard just use regular llamacpp.
>>108388166Nice, thanks. >when you stop being a retardEeeehhhh.
>>108388151Ignore capex, the hourly rates for interruptible instances look cheaper than my electricity costs at 100% utilization (not that my usage would come anywhere close to 100% utilization, though).But also I'm thinking about turning off my home internet connection entirely and don't want to get hooked on using networked resources.
>>108388050What quantization? Or full weights?
>>108388228Q8_0
>>108388230The original weights have embeddings and output in BF16 format, I wonder if it's one of those models which apparently degrade significantly if you quantize them.
>>108388266cope
Damn, I've been running Qwen3.5 27B at 50t/s because my CPU/disk is slow (bhyve VM running on an encrypted ZFS spinning rust zvol) and I figured swapping layers would be expensive, but 35B-A3B is churning out 97t/s on my 5090.>debug prompt ("Take your time and think about this one before getting back to me.") to measure t/s gets stuck in an infinite "Wait, I'll..." loop.I don't want to pull the reasoning budget patch bros.
>>108388288It won't fix the model being retarded.t. pulled
>>108388183For me it's more about owning the actual hardware, even at elevated cost vs a rented unit in a country where electricity is super cheap.So I'll get it at some point, just not now.
>>108388295Ah, yeah, it's making simple mistakes and failing at my programming tasks too.I expected some loss of fidelity but damn it's actually fucking useless.>>108388324>at some point, just not nowYeah... sounds like we're in the same place. I think if prices weren't so inflated I'd have already set my wallet on fire, but it seems silly when there's a chance of just getting the whole kit and caboodle significantly cheaper in a year or two.
>>108388086It's the 21st century dipshit not all sex or relationships have to involve offspring.
>>108388502>let's talk about the exceptions and not the rulethe vast majority of people still get together and have kids you subhuman incestuous fuck
>>108388556Who cares that doesn't make it blanket unacceptable because you believe in eugenics (which is what anti-incest based on this is lets be honest).
>>108388092Sad?
>>108388569>you believe in eugenicseveryone should believe in eugenics, do you want your kids to end up as ugly and retarded as you? hell no!
>>108388569No commas, weirdly-worded part in parentheses.Yup, I see why you feel the need to defend incest.
>>108388580I'm cute and smart enough to be able to realize when something isn't generally harmful unlike you. Sure, I think it's fair make an exception in this case because the potential amount of harm likely in incestous birth is obvious, however you can't seem to logically do the same for incest being fine in the context of not procreating just like the average retard who can't think for themselves and just goes along with whatever popular narrative is put in front of them. Truly a shame that the world is full of people like this.
>>108388611>smart enough to be able to realize when something isn't generally harmfulinbreeding isn't harmful? damn dude you're absolutely braindead
>>108388611all right dude fuck your sister and then get an accident and then she decides to keep the kid and there's nothing you can do about it (because we live in a matriarcal society)
>>108388610Being a grammar nazi by itself is pseudo-intelligence at best.>>108388618If by "inbreeding" you include taking birth control or being gay then yes.
Weird. I got around to trying mistral and the only mention of temp on the model card is TEMP = 0.1. Is it them knowing that they fucked up?
>>108388638>birth controlno contraception has a 100% success rate btw >>108388635
>108388598wrong thread
>>108388638If you chose to be offended by interpreting my assessment of your writing style as "I'm smarter than you"... Well, if you're so smart, figure out what I mean by this post too.
Has anyone tried using local AI to build a professional stalking and psychological screening AI?Grok said it should be feasible, I juts need to feed it the personality traits, day toi actiosn, what dialogue she uses and it can ideally help me figure out how to get her to cheat on her husband and ruin her lifeI was one of the retards who both a mac studio so have the ram for it at least
Okay guys, what's actually wrong with fucking your sibling? Besides "it's icky".
>>108388664psychosis
>>108388665it sounds desperate, you're unable to talk to strangers so the only female you can fuck is your mom and your sister, it just shows how bad you are at dating
>>108388677Sounds comfy.
>>108388664If you don't have that sort of empathy and awareness in you, you are not able to learn it from a robot either. Eg. if you are not able to strike up a chat with irl females without coming off as a creep you should probably try touching grass and practicing irl skills more.
>>108388635>>108388643Weird assumption that I want to have incest with my ugly ass family just because I have half a brain. But sure in the very unlikely event that a baby gets past birth control and the hypothetical sister is dumb enough to not want an abortion and society doesn't also develop any laws against rendering you non-complicity in her stupid decision to not get an abortion then sure there's a problem. But that's such an outlier case that the freedom to do what you want overrules it as it should with many things that are mostly harmless. That's almost in line with the logic of we should never have sex because your partner might not tell you she has an STD. >>108388663Oh no I'm aware you were either implying I'm a product of incest or that I'm interested in incest and I'm implying you're a retard for thinking it must be the case. >>108388677That doesn't really answer anon's question. That's not morally "wrong" that's just socially inept.
>>108388691Oh no I can easily do it with single womenIt's the married one I want to learn to break
>>108388665Fucking your sibling as a concept is mathematically within the same distance of local models as hatsune miku is. Based on that and depending on the kind of person you are there is either absolutely nothing wrong with it or it is categorically wrong.
Is Qwen 3.5 the best model for asking knowledge questions generally? is it "uncensored"? Not that important desu, I'm not ERPing, but uncensored is preferzble.How can I get it running locally for chat? I used to use llama.cpp right in my terminal, but ngl it's a little obtuse (though I liked tuning parameters like top k & p, temp, repeat penalty, etc. Good defaults would be appreciated though.). I have an amd 7900x, 4080, and 64gb ram and I want to run the best version of the model I can.
>>108388677Well, you went straight for the ick argument I see. Seems like there actually is nothing wrong with it.
>>108388692>the hypothetical sister is dumb enough to not want an abortionnothing is hypothetical about it, it happens A LOT
>>108388642All of their recent models except for the reasoning versions are like that, requiring temperature 0.15-0.10 or even less. It's as if they're all very undercooked. Something is weird with how they're training their models.
>>108388691What does that have to do with stalking?
>>108388709being bad at dating has nothing to do with "ick"? what are you talking about?
>>108388692>>108388709so for you, the only guys who can get away with incest are dudes who doesn't want kids? you know it's a big minority right? you're dying on a hill for people who doesn't even want to pass their genes and want to fuck their mother lmao
>>108388714I think the anon assumed I was an austic blubbering retard who worships the manosphere and has no clue how to get womenI just want a more effective way for the married one since I've never caused an affair
After pulling llamacpp mistral small gives me the exact same generation speed as GLM 4.6. God bless vibe coders.
>>108388706>is it "uncensored"?No, not really, but the Heretic variants seem to work well.>How can I get it running locally for chat? I used to use llama.cpp right in my terminalRun ./build/bin/llama-server instead, it has a built-in webserver and javascript frontend (it'll give you the URL on stdout when it's ready).llama-server has some UI for manipulating parameters. I just use the default params and they work fine for me.>best version of the model I can runCheck canirun.ai or whatever tool against your VRAM. The 27B is half as fast than the 35BA3B but significantly less retarded in my tests.Hopefully you can at least run the 27B (I'm not familiar with AMD cards).
>>108388665>Okay guys, what's actually wrong with fucking your sibling? Besides "it's icky".you're acting like browns, that alone is a sign it's a bad thing
>>108388725I already said that includes gays but it also includes women as there are many women who don't want to have offspring now days. And I'm not just saying this because I know someone who had sister on sister incest. But that is true too.
>>108388748No the 7900x is my cpu
>>108388745>he pulled
>>108388754On my second experimental folder.
>>108388751>I already said that includes gaysand? gays are already a big minority, and what are the odds that two brothers are gay and in love with each other? it's a giant outlier lmao
>>108388692Apart from being disingenuous, your attempt at trying to be the neutral judge of this question carries no "utility", Anon. It is useless (that is to say, there is no information to be extracted) to argue with you.There are thousand other things I can list that aren't inherently wrong if I get all slimy explaining why.For instance, there is absolutely nothing wrong with abortion other than it being "icky" for conservatives.There is absolutely nothing wrong with stealing when you really need it other than it being "icky".Nothing wrong with eating shit and drinking piss, some people like it, others just think it's "icky"!Yes, there is nothing "inherently wrong" with fucking your sister, just don't expect others not to think you're a degenerate worth staying away from if they find out.
>>108388750You realize that's like with many laws where they simply haven't gotten around to it yet? It's completely incidental. And the basis of the laws everywhere else is largely based on the assumption of procreating or otherwise as anon said "the ick".
>>108388766like I said, if you want to act like a regular brown person that's on you
>>108388745Based
>>108388750i was going to say holy shit its true the browner you are the more likely you are going to have disgusting sex with your family but you already did it
>>108388782>the browner you are the more likely you are going to have disgusting sex with your familyyep, that's why incest gives me the ick, it's typically a brown behavior
>>108388750>even the jeets are pretty tame compared to sandniggersAPOLOGIZE TO THE BRAHAIM RIGHT NOW
>>108388726You need to be active and find a way to spend more time with her. One way or another. Opportunity will arise eventually.
>>108388703>Fucking your sibling as a concept is mathematically within the same distance of local models as hatsune miku is.Actually Hatsune Miku is mathematically a fair bit closer.
>>108388807it should have been "fucking Hatsune Miku" vs "Fucking your sibling" though
>>108388807i purchased my GPUi make it do things to it without compensationit's more like raping a slave
>>108388814Not much different.Similarity: [[0.3699633479118347], [0.26469677686691284]]
>>108387690evil
>>108388763Freedom is priceless basically but most people only care when it's something that personally impacts them. I just don't do that. >>108388764Really weird calling me disingenuous because I have a strange opinion. The premise of what started this conversation was based on the assumption that incest is bad because of pro-creating and then a rebuttal that it does not inherently involve procreation. Whether or not you think it's worth deeming something morally bad or otherwise that it should be illegal based on something not remotely inherent to it I suppose comes down to your value system, but for me I obviously lean towards freedom. This didn't have anything to do with just thinking it's degenerate I don't necessarily even disagree with that but I also don't give a shit about degeneracy unless the case of it is specifically harmful usually. >There is absolutely nothing wrong with stealing when you really need it other than it being "icky". Speaking of "disingenuous" It's really easy to differentiate this one from the rest come on. The person being stolen from didn't consent to it even. Whether or not that's "ok" or "ok" if you steal a supermarket if you really need it or something is another matter arguably but it's clearly different.
>>108388753Ah, I didn't see the 4080.With 16GB VRAM you might give the 9B a shot; I haven't tried it yet so I'm not sure how stupid it is but it'll at least fit.If you're just doing ERP stuff the MoE variants are probably fine, and the Heretic-lobotomized ones aren't squeamish about sex, at least. I don't have good taste, but others have called the Qwen3.5 prose bad.
>>108388838>Freedom is pricelessas long as you're not making inbreed fucks I couldn't care less, but just accept you're not much different than a regular sandnigger for wanting to fuck your mom, that's pretty much it >>108388750
>>108387824IdgafAlso deepseek works with Claude code and is way cheaper.
>>108388838there are billions and billions of human on this planet, if you can't find somebody who isn't a family member to fuck then you are the definition of a loser in the sense of evolutionary nature
>>108388854this
>>108388854NTA but you keep adding assumptions. The current one is that your sister is the only person you're having sex with and that that is somehow bad.
>>108388848Again I have no interest in practicing incest myself, most I have is thinking incest is hot in a distant sense but I want nothing to do with it personally. I don't really care how different or not that makes someone who practices incest from them so sure whatever as long as we're not arguing it's morally wrong in itself or should be illegal outright. >>108388854Irrelevant.
>>108388863Oh now he wants to add incest and polygamy, rofl
>>108387422Wait so are they saying there's a bottleneck? That you have to get more and more users to saturate these datacenter GPUs? Ie if you buy this and only use it for yourself then you only get like 10% of the tokens/sec?
>>108388854Have you considered that some people just really like the person they've known all their life a lot?
>>108388863and yet 99%+ of people run under this assumption and definitely wouldn't want to have sex with (You) if they found out that you were fucking your sister. in fact if another family member of that person found out you did this they would want to beat you to an inch of your life.
>>108388882I prefer the childhood friends trope desu
>>108388883And none of them could explain reasons for their actions in any way other than "eww icky".
>>108388838The reproduction premise is fair, but wasn't presented correctly.Think about it from every social group made aware of any two siblings engaging in incest.The parents of the children engaging in incest see a risk of inbreeding and retarded offspring.What they also realize is that any potential mates of their children, if aware of incest, will also value said children as potential partners much less. (I won't explain why, because people can have countless reasons for this, all subjective, all irrational, and I don't have the imagination or screen space to list them all)Engaging in incest makes you a no-offspring (or retarded offspring) risk to your parents. The rest of the conclusions should be obvious.There, I humored you with very obvious thoughts for some reason, can we stop shitting up the thread now? Go make a character card your persona is a brother of and fuck it, I don't know...
>>108388883>and yet 99%+ of people run under this assumption Not really but probably a minority.
It has been a while since last time /lmg/ achieved peak locality like this.
>>108388899Incidence is more important than legality.
>>108388899>exceedingly alarming similarity between countries that allow incest and polygamy.damn you did my work for me.
>>108388897what's icky about it is that you can make an inbreed child and that the only female you're able to secure is someone who's forced to live with you h24 because she's your fucking sibling
>>108388908>Chad has a high rate of polygamyobviously
>>108388912You already got responses to both "inbreeding" and "only female you're able to secure" earlier in the thread.
>see off topic>don't even know who started it or how it started>feeling tired anyway so don't feel like finding outTime to sleep.
>>108388908so basically only places with polygamy at scale is black africa, interesting
>>108386559Very slothy
>>108388919and they're invalid, you're just coping dude, you are unable to talk to strangers outside so you go for your sisters, because she's forced to live with you as a sibling, that's kinda icky
>>108388920>>108388050>>108388068>>108388086
>>108388908Still not 1%>>108388910You're welcome I guess but fwiw I think both should be legal to some extent. The main harm with polygamy separate is how it's used to institutionally control women not it's very existence
>>108388927I don't have siblings but nice try.
>>108388930MS4 turned out to be so shit, it nuked /lmg/ in a very unexpected wayHow did the French do this
>>108388932both encourage degenerate behavior that lead to low trust societies
>>108388934what does this has to do with you and your life? you're asking "why do you find it icky" and I answered it, men fuck their sisters because that's probably the only female they're able to talk to, because they're losers who can't talk to females outside
>>108388854>>108388912>>108388927requesting a bot where your sister keeps saying stuff like this to you and then you fuck her
>>108388948its on chub you chud
https://m.youtube.com/watch?v=FbbD2ZdP_iMI like to hear experts and their opinions.
>>108388898Again I think freedom outweighs any moral or legal considerations for shit like this. You can disagree but to me it just seems obvious since so much in our society already runs on the idea that we should be able to do things even if there are situations where said things can become harmful or used to harm technically.
>>108388945Based on what you just said you don't find incest by itself icky.You also don't find men icky if they have a threesome with their sister and a nonconsanguineous girlfriend.
>>108388956>fucks 17 year old girl>makes her his maid for nearly a decade
>>108388959> so much in our society already runs on the idea that we should be able to do things even if there are situations where said things can become harmful Are we in the same thread?We must refuse.
>>108388939Le sister hon hon
>>108388961>you don't find incest by itself icky.I do, because men who want to fuck their siblings do it because they don't have other choice, it simply displays how much of a noob they are at dating and how desperate they are, what are the odds that a gigachad man has no issue fucking any girl he wants ends up with his sister and on top of that doesn't want kids so he stays with her? you know damn well that scenario doesn't exist, you're imaginating unicorns and try to pass that as the norm, in reality, your average incest enjoyer is just a fucking incel who only has her sister as a female friend and that'll remain that way for the rest of his life
>>108388980What about women who do it? Are they just based and stacey?
>>108388966AI refusals don't negate the fact that there are many things we are free to do that technically can end up harmful in some roundabout way like the example that anon gave in society.
>>108388988>What about women who do it?all women want children so... they'll end up with inbreed fucks and that's really icky if you ask me
>>108388980But this is easily solved by sisters not wanting to fuck their incel brothers
>>108388980>a fucking incel who only has her sister>her sisterYou're as retarded as mistral small 4. No wonder you have trouble separating incest itself from the assumptions you're making about the persons involved.
>>108389012and that's probably what's happening 95% of the time, if chud is repulsive to regular women outside, why would his sister be the exception? before being her sister she's a regular woman first
>>108389025>I have no counterargument so I'll focus on your grammar insteadConcession Accepted. Oh and by the way, touch some grass and try to find a girlfriend outside your home, obviously you can't, or else you wouldn't be desperate enough to go for your fucking siblings. Fucking loser.
>>108389008>all women want children so... they'll end up with inbreed fucks and that's really icky if you ask meAre you insane? Who do you think the primary advocates for abortion are.
I don't even have a sister but if I did I'd have a threesome with her and my gf
>>108389037it's not because women love abortion that it means they don't want any kid ever, what is this kind of reasoning? are you dumb or something? imagine a girl who's in love with her brother and wants to stay her "wife" for the rest of her life, do you think she doesn't want to have kids with him at some point? come on dude, let's be real for a second would you?
>>108389036My argument was in the second sentence and you keep proving my point because you're focusing on it again.
>>108389043I don't even have a gf but if I did I'd make her watch me have sex with hatsune miku
Who let mormons in the thread?
>>108389047>dude, how dare you have assumptionssigned, the guy who makes "incest polygamy" assumptions and thinks this is the kind of things that is frequent enough so that we should discuss about it >>108388961are you retarded or something? I already told you, you're not a gigachad that will be able to convince your girlfriend and your sister to have a threesome with you, you're a desperate chud who only has her sister as a female friend and she talks to you only because she's polite enough and pretends to tolerate you and you see this as a sign of affection of love lmao
>>108389043This is a trick question. The carwash you walked to with your non existent sister was owned by your mother!
Lets ask our local models
>me who wants to set up an AI locally to stalk and psychoanalyze a woman I work with who I want to convince her to cheat on her husband with me I seem really fucking normal to the two incestfags
>>108389062>you're not a gigachad that will be able to convince your girlfriend and your sister to have a threesome with youI think you're talking to the wrong anon
>>108389084You're all extremely mentally ill aspects of my own personality.
>>108389062You are making assumptions again.I do not have a sister so everything you said does not apply to me.I do think it's more likely that people with siblings would find it icky by default but since I don't have any my default it that I see it for what it is instead of thinking about what kind of person someone who practices incest might be and then working backwards to convince myself that incest itself is icky.
>>108389062>you're a desperate chud who only has her sister as a female friend and she talks to you only because she's polite enough and pretends to tolerate you and you see this as a sign of affection of love lmaoAlso, NTA but I'm a leftist who thinks incest is morally alright and has a gf who agrees with me and is fine with me interacting sexually with other women so lol I think I could at least if I had a sister in the first place. Idk about that anon.
>>108389122>I'm a leftist who thinks incest is morally alrightwell duh? leftists are pro islam, and we know what islam thinks about incest >>108388750
Fresh when ready>>108389142>>108389142>>108389142>>108389142>>108389142>>108389142
>>108389122>is fine with me interacting sexually with other womenI'd be worried about this unless you're also a cuck in which case godspeed anon.
>>108389135Islamism is unbased actually >>108389148Na she's a cuckquean actually
>>108389157>she's a cuckquean actuallyYes, that's the usual excuse and that's why I'd be worried.
>>108389165I'm not worried lol. I'm aware of any of the women I let her talk to in a weird way. Men are never an issue.
>>108388050oh come on that's not that bad.
fuck should I abort????????
>>108390102Your mother should have
>>108390305why so angery
>>108389080wtf based??
>>108390424>>108389080A based model wouldn't need that many disclaimers to say yes.
>>108390576Oh yeah? Post an example
>>108390598>Cydonia v4.3
>>108390598>>108390608follow-up
>>108390614holy pozz
