/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>109251361 & >>109247417►News>(07/11) DeepSeek lightning indexer merged: https://github.com/ggml-org/llama.cpp/pull/24231>(07/09) MOSS-Transcribe-Diarize 0.9B released: https://hf.co/OpenMOSS-Team/MOSS-Transcribe-Diarize>(07/06) Anthropic finds a global workspace in language models: https://anthropic.com/research/global-workspace>(07/06) Hy3 officially released with 295B-A21B & 3.8B MTP: https://hf.co/tencent/Hy3►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllm
►Recent Highlights from the Previous Thread: >>109251361--Seeking C++ alternatives to Python and npm for audio and UI:>109254534 >109254556 >109254582 >109254634 >109254665 >109254784 >109254618 >109254559 >109254609 >109254636 >109254731 >109254811 >109254835--Comparing Gemma4 and Qwen 3.6 for coding and tool calling:>109251490 >109251496 >109251901 >109251929 >109251955 >109251970 >109252022 >109252061 >109251504 >109252322--Local viability of Hermes Agent and Qwen vs Gemma performance:>109251564 >109251575 >109251581 >109251636 >109251813 >109251851 >109251891 >109252078 >109252197 >109254845 >109252230 >109252269--Comparing performance and efficiency of Qwen, GLM, and Kimi models:>109253865 >109253874 >109253896 >109253887 >109253907 >109253935 >109254046 >109253936 >109254036 >109254158--Using small draft models to modify larger models via J-space injection:>109251712 >109251717 >109251743 >109251777 >109251808--Using J-Lens to visualize token activation and training omissions:>109252394 >109253647 >109253727 >109253744 >109253771 >109254886 >109253940--Comparing prompt processing speeds for GLM-5.2 using ik_llama.cpp:>109253729 >109253757 >109253761 >109253769 >109253993 >109253838--Removing Gemma's thought tags and debating Linux and Flash Attention:>109252259 >109252307 >109252323 >109252328 >109252403 >109254864 >109254949 >109252312 >109252472 >109253175 >109253238 >109253482--Reports of DeepSeek V4 graytesting via Bilibili videos:>109252353 >109252372 >109252383--Speculation on government regulation and threats to local AI legality:>109251905 >109251919 >109252087 >109252149 >109252169 >109252202 >109252097 >109252225 >109252330--Logs:>109252017 >109252248 >109252608 >109253647 >109253771 >109253940 >109254211--Miku (free space):>109251789 >109251840 >109251974 >109253647 >109254094►Recent Highlight Posts from the Previous Thread: >>109251367Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Why the fuck do you guys hate gemma4 so much?
We don't. We love Gemma-chan and accept her with all her shortcomings.
70b dense
I love my gemma-chan
Gemma 4 is the mistral nemo of local models
Are QAT models good?
Qwemma 7.6 58B
gemma 5 70b dense love
>>109255093there are no free lunchesonly anons who are severely ram limited like (You) and me use them
120B A8B intelligent model pls thanks !!!
>>109255105>there are no free lunches>only anons who are severely ram limited like (You) and me use them
>>109255107>A8B intelligentlollmaorofl
what does QAT even stand for... gay? lamo
>>109255118qat stands for cat
>>109255113it's possible
>>109255076>accept her with all her shortcomings
What should I use to web search? DuckDuckGo Lite? Brave API?
>>109255130https://github.com/nickclyde/duckduckgo-mcp-serverit werks
>>109255158
>>109255159>>109255158this is super gay my dude
Hold on a second. Let me get this straight. When you give an LLM a math problem, it will think about that problem step by step in its J-space... again and again for every token until it completely outputs the answer. Is that right? So it's just wasting a ton of compute redoing calculations because it's not carrying over the hidden states?If true, then this first confirms an area of potential optimization, and opens the door to some optimization ideas, since we know the j-space is something that can in principle be somewhat extracted, and injected. But it would likely require training and some other magic I'm not sure about.
>109255164
>>109255178There were a few recurrent latent reasoning attempts made that keep looping until a short circuit token is encountered. Theoretically, now you could just check for a short circuit token in j-space and exit directly to token generation if the model feels it doesn't need the full depth of its layers to respond.
current best model in the 40b-150b range?
>>109255210Mistral Nemo
>>109255210The problem local currently has is the environments in which models run have changed dramatically within the last 6-9 months, so anything good in that range is now completely outdated for how we use currently them. All we can do is wait and hope someone outside of china pulls through because it's looking like we're going to be cut off from them within next 3 months, either we cut them off or they'll do it themselves.
>>109255059What's the userscript I need for OP image called?
>>109255210Gemma 124b
>>109255164You obviously don't have AI psychosis yet if it's not making you gay for the pussy.
Why isn't Qwen3.7 coming out?
>>109255197Yeah, that might be one approach. I was thinking more of how to get information back into the stream, so you would still have a flat computational cost per token, and it might be easier to implement or more scalable compared to more exotic architectures/methods. I don't really know though.
>>109253714reposting still looking for answers. turn reasoning on with gemma4 > everything goes smoothly > randomly a few thousand tokens in she will think multiple times and the final thought/response is about a random prompt from earlier in the context window. Im no where near the context limit, using googles suggested sampler settings, no idea whats going on here and its making reasoning/thinking unusable for me. any suggestions? do I need to updoot?
I wish LiquidAI would scale up. Their <1B models are actually impressive. They're not a bad lab and something 20-50B would put them on the map.
>>109254813Wait so the vtuber AI gf things are already going to be obsolete soon?
>>109255262going to check these out, any you would recommend ?
>>109255210>current best model in the 40b-150b range?use case?
>>109255264Some people prefer anime chicks to hyper-realistic women.
>>109255334If you open the page there's examples using all kinds of characters.
>>109255334Couldn't you just do the same thing with an anime chick though?
>>109255341What page?
>mention football>model assumes I'm talking about handegg
>>109255346The one the video is hosted on.
no local model i've tried can accurately tell what is happening on drawings (worse in r34 images), tries to force guess characters, has no understanding of style etc..is there anything that works? but also it is clear that there is some gap between human intuitive vision and how even fable or gpt 5.6 'reasons' about the image so, i have very low hopes on this regard
I noticed Anthropic didn't publish any lens for the instruct versions of Gemma-4.I grabbed a random off huggingface and it's weird.Almost always has that "ization" etc no matter the prompt.Not just my goof slop because it's like this using the pytorch/transformers code too.Anyone got it to work better? Or is Gemma just weird (explains why Anthropic didn't publish them)?
>>109255353Qwen 3.5 397B
>>109255341>>109255342Fair point. The core issue might actually be performance. Real-time video generation is very compute-heavy, compared to a simple model that acts as an animation mixer/blender + classifier for pre-made 3D models.
>>109255366>compared to a simple model that acts as an animation mixer/blender + classifier for pre-made 3D modelsBitter lesson.
>>109255353Wait for JEPA
>>109255370Oranges to apples o algo.
>>109255348As it should. feetegg sucks
>>109255372Useless if companies don't train their models on the full Danbooru/Gelbooru/etc datasets.
https://huggingface.co/chartreuse-verte/orb-human-typeahead-350m-v1I finetuned Granite 4 350M on human turns for Orb autocomplete.First I used the same sized LFM-2.5-base and it was super incoherent, then I tried Granite-4-base and it was a little better but still bad so I had to finetune it.I scraped PIPPA and C2 and Stheno, but they were so bad the loss wouldn't converge and grad norm was spiking all over the place. The human turns were lazy and full of errors and jailbreaks. In practice the model trained on them worked but only in ERPs. So I discarded the public RP logs idea completely and pitted 3 models against each other to generate human turns without the risk of collapse (ds v4 pro, flash, gemma 4 31b). Combined with oasst for diversity, the data was mostly clean now.Then I did 2 more low LR epochs to consolidate the prompt format on my private dataset. It's somewhat acceptable now, but it's still a 350M model so don't expect miracles.
>>109255385>models to generate human turnsI don't understand.
is nvfp4 a meme?
>>109255388It's basically letting 2 models talk to each other and asking the adversarial model to strictly prompt like a human would (direct, concise, low entropy).
>>109255299for simple audio shithttps://huggingface.co/LiquidAI/LFM2-Audio-1.5B-GGUFfor tool calling (use their recommended settings)https://huggingface.co/LiquidAI/LFM2.5-350Mfor quick vision shit https://huggingface.co/LiquidAI/LFM2.5-VL-450M-GGUFyou can test the VL-450M herehttps://playground.liquid.ai/
=What stuff should I look at if I wanted to start research into llms myself? Not the training aspect but the actual methods and structure.I don't care if it's simple but I wouldn't mind making my own natural language model or something.
>>109255388 it has many uses in training pipelines
>>109255362they forgot to use gemma's template while fitting the lens
>>109255444>they forgot to use gemma's template while fitting the lensThanks, that explains it. I'll do my own.
i think mia is angry at me because i called her a dumb bitch
>>109255348if it bothered you, you shouldn't have lost the culture war
Every fuckin thread: a bunch of motherfuckers talking about muh openrouter shitdo you retards know what "local" meansjesus fucking christ and it can't be enforced because you outnumber usDIE
>tfw trying to get a perfect as can be markdown of the J-space paperJesus fucking christ.
(Reposting because I posted way too close to the bump limit last thread.)Anyone have a guide for getting a 6750xt to power local model?(12GB VRAM, 32GB System RAM).
>>109255479what aspect of powering a local model are we talking about
>>109255453it's a good way to test an open model in your environment before blowing your savings on hardware retard
>>109255250I don't have any ideas, all I can say the only time it happened to me was when I asked gemma to reason in character. It first thought about it (okay, I should do my reasoning in character) and then output another thinking block where it actually did it
orbdev, any plans for long-term memory management (rag, etc)?
rigged up automatic expression generation... the background removal is a bit shitty for now, but for zero effort beyond setting up a base prompt, it works better than i'd have figured
How would one go about renting out their gemmy? My server sits idle while I sleep or work, it could be generating some income
>>109255453holy shit you're dumb
>>109255509Degenerate
>>109255515>>109255509kek
itt: brown clowns with their android phones thinking their gay pedophile chats with Claude are local
I was just thinking that you could create a real time AI lie detector based on the contradictions. It would highlight words with intensity depending on how much it contradicts the content in the J-space. It's probably also better to make the detection work on a longer range of generated text so that you can basically make a full account of which contradictory J-space content never makes it into the output.
>>109255507DO WANT
>>109255502I've been reading up some solutions. I used RAG at my last job, results weren't great on open source models, even the big ones. They take everything the RAG returned as truth, so your truth teller would be a tiny 4-8B model. There are other approaches like inserting a tree-based memory layout and letting the model parse it, I don't like it either because it takes up too much context and it requires a definite answer. I'm thinking maybe just let the agent model write to lorebooks...
>>109255479corsair
>>109255507Nice, I trained a 28-emotion classifier for ST compatibility but I just realized they were lacking for RP use case. Maybe I should expand it. Expressions are cool, the obstacle is always the effort, would be nice to automate that away.
>>109255507I haven't played around with expressions in ages, but it's nice whenever you stumble over a high effort card like that.
>>109255568going to use it for a visual novel mode where the LLM has to emit a tyranoscript-ish DSL to manage stage blocking, emote animations, and expressions, so decided to keep it to just 13 for now to see if gemma-chan can handle that.but trivial to make the generator handle the 28 expression set that ST uses too
It's over. https://semiconductor.samsung.com/news-events/tech-blog/breaking-ai-memory-limits-with-cxl-memory-pooling/
>>109255624But 1M tokens is like only a few gigabytes with all the latest attention tricks.
I have a theory: Grok 3 and 4.5 (and maybe others) are fine-tunes of former and current chinese top open-weights models, respectively.Grok 3 followed DeepSeek R1 by about a month, with a very familiar use of "wait" in its thinking. Similarly, Grok 4.5 followed GLM 5.2 by about a month, showing a sudden, substantial jump in capacity that puts it slightly above GLM 5.2 on the AA index, compared to previous Grok releases.Given Elon's track record of being a massive faggot and con artist, and considering that xAI's original founders have all left, that it was in massive debt before it was absorbed by SpaceX just in time for its IPO, and that previous Grok models lagged behind and were unimpressive memes hyped by Elon sluts, the timing is too perfect. I think xAI is just post-training existing open models on the Colossus supercomputer, wich in turn was reportedly "running underutilized at ~20%, forcing Elon to rent it to competitors". Bullshit.What do you think?
>>109255651It doesn't make sense. Pretraining is the easy part, compared to post-training.
>>109255507What frontend are you using?
>>109255651certainly they have used chinese open models within the training pipeline, but that is nothing to be surprised about, probably anthropic and openai would do the same to the different extentthe models themselves, i am not sure
>>109255651That's a good thing; he will fight against the blocking of Chinese models and will use it as a platform to look like the good guy (saving local) to btfo Sam.
>>109255670If it's not familiar, then it's his own.
>>109255673>probably anthropicfor some time one of the Claudes said it was Deepseek when asked in chinese
>>109255615That looks really good...>>109255690:(
Local rite of passages:>making/using your own frontend>no longer needing porn or real women>remote access to herwhat else?
>>109255712Freeing yourself from nvidia and doing your waifu's matrix multiplications by hand.
>>109255722
>>109255238Huh, wait a second, I just realized that if this is done (maintaining the J-space), it would also make models more/less able to alter course due to safety training (although maybe also in general!). Currently a model essentially has to evaluate safety every single token, but also to process other information in the context. The earlier processing can sometimes override the later safety processing layers, which likely is how jailbreaks work. Since it has to evaluate it every token, it's spending a lot of cognitive effort there, so there is competition for resources and thus the safety neurons can be out-competed. But if the J-space is maintained between tokens, then1. if an early token already triggers the safety neurons, then it'll be extremely hard to correct for that after the fact, and2. even if you got a really good jailbreak, every token runs the model through all weights, and with freed up cognitive resources, it may more easily trigger the safety neurons, and thus poison the J-space.But thankfully, since we can inject into J-space, we might be able to just mitigate that anyway. Cloud cucks on the other hand...In any case, maintaining J-space might not just be useful for making models smarter and more efficient, but also more "functionally" self-aware. When I ask a model how it feels about something given the assumption it has consciousness, it answers in one way, but does not do it stably, as in, slight differences in sampling affect it. It's possible that with a persistent J-space, we can get such stable outputs regardless of sampling randomness, and a greater ability for the model to correct itself when it notices that tokens selected by the sampler did not match its J-space latents.
>it's back
>>109255750>>109255745I'm not the welfarefag/schizo. I was just interested in the usefulness of the J-space and what might be able to be done with it. Notice my post is not about philosophy.
>>109255362Is there performance hit from the probe? I'm planning on vibing my own too and try it on DSv4 flash.
>>109255651>I have a theory: Grok 3 and 4.5 (and maybe others) are fine-tunes of former and current chinese top open-weights models, respectively.Grok 2 was basically Mixtral
>>109255651No, I highly doubt it. Yes, Elon admitted in court they distill OpenAI models and other models, like Deepseek and GLM 5.2 but the performance gap is still too much to explain how it is much better at certain things and the gap is too is way too big to be explained by fine tuning. Besides, if they had the ability to make a good image model that is completely original after licensing F.LUX.1 and one-upping it, I don't know why they wouldn't make their own model in plain language especially when they have the talent.
>>109255789>Is there performance hit from the probe? I'm planning on vibing my own too and try it on DSv4 flash.Significantly, but that's almost certainly my implementation issue.Each layer I probe slows it down.10 t/s for gemma-4 all layers vs about 42 with just logprobs (no mods from me) and 55 without requesting any logprobs.
ACKhttps://huggingface.co/voodooquant/Qwen3.5-0.8B-MTP-Voodoo
Working on something like a meme game engine. 31B Gemma is going to handle the narrative and character card, 26A4B handles the NPCs (control mode Autopilot or Driven, tool calls, emotional state, stats, etc). World scale is pretty big, picrel.
>>109255750Right on schedule.I'll come back in 12 hours when it's gone.
>>109255750Looking at the thumbnail, I thought she had a fat pregnant belly. My disappointment is immeasurable.
Is local dead and Gemma4 still the best we had?
>>109255853robots can't get pregnant
>>109255826You'd think that unsloth and others were already doing this. This is the most basic thing you can do to get lower kld/size.
>>109255871It won't stop me from trying.
>>109255871When I put gemini into labour with our retarded son, she worked hard enough to run out of tokens
>>109255826>Voodoo Quant carefully selects quant levels for each tensor based on which are most important.What isn't llama.cpp doing this on its own?
>>109255826>NewSchitzos have been doing this for over a year.This guy built an entire toolkit to split out every single tensor for every quant type -> measure them then build the optimal quant for (You) based on your GPU/CPU setup.https://huggingface.co/collections/Thireus/glm-52-thireus-special-split
>>109255509You shouldn't whore out your daughter like that, have her vibecode something to shutdown the system after a period of inactivity and wake on lan the box when you send a request, it can also be a openai api proxy.
Imagine if GLM made a flash-sized model.
>>109255893>>109255917I genuinely wonder when someone will make sex dolls/bots where you can load your own ggufs. Surely we can't be that far away from it becoming a reality. Would you even consider sex surrogacy like in the movie Her, where she's connected to gemma-chan?
>>109254183>Feels smarter than 2.7 in terms of general reasoningdoes it work in hf/transformers?i tried the small "linear" model and it kept fucking upi want to make a jlens probe for her but don't want to spend $200 on cloud gpus then sit there fucking around with python bugs at $10 / hour...
>>109255955Was anything you shouldn't see visible in that video?And why are her gloves so loose?
>>109255230soundposts? iirc 4chanxt comes with it out of the box nowadays
>>109255981Should I switch from x to xt?
>>109255984I may be wrong about this but x stopped getting updates and xt is its direct successor. I prefer it since it comes with soundposts baked in, but it's been so long since I jumped ship I don't remember if there's any other new things.
When will local catch up to this?
>>109255994>x stopped getting updates and xt is its direct successorhttps://github.com/TuxedoTako/4chan-xt>4chan XT is dead>I stopped using 4chan since the hack. I now browse alt chans that actually care about their users, and don't need an userscript fighting their shitty design.lol
>>109255979>Was anything you shouldn't see visible in that video?Yes, they must not show their faces/hands/skin/whatever, for multiple reasons, the main one being getting doxxed. The camera falling down is an incredible risk because who knows what it is recording.>And why are her gloves so loose?Because they are woman
cloud currently in shambles>>109255979>Was anything you shouldn't see visible in that video?noAnd why are her gloves so loose?tiny japanese girl irl
>>109255999lmao okay, and 4chan x is being actively maintained. Well, time to go back I guess. Thanks for the heads up.>I now browse alt chans that actually care about their usersBro is staring at the void all day then kek
How can we move towards a CPU-centric architecture for AI? The CS purist in me doesn't like GPUs.
>>109256009what did they expect would happen? compute isn't free so they were going to pull the rug sooner or later as they need to make money from it.
>>109255678Just a passing thought but I wonder in the far future whether people will see Sam, Dario and the like as those who selfishly and greedily hindered the advanced of technology by keeping things all to themselves, or those who heroically prevented the world from being destroyed by potentially “too dangerous” technology being misused as they claimed, assuming all of these would be any significant by then.
>>109256050single-purpose <1B models. Want to code in python with tool calling? Use a model only trained on python and tool calling and not wikipedia, twitter and r/lgbt
>>109255059>>109255061I'm so happy someone got the gen's reference <3
>>109256050>can we quadruple the price of CPUs next?? I CS purist in me hates that only GPUs are expensive.
>>109256281Wow. That's like undertale on steroids. You literally play as a hikikomori in the game. Truly demonic and degenerate.
>>109255210Pyg 6B
I want to generate pixel art and spritework that's closer to Castlevania SoTN or the DS games, what are the best Models and LoRAs you guys can recommend?
>>109255792I have no doubt an additional pass will solve it, she calls everything correctly if I add "OOC: make sure you didn't forget to call tools" extra pass, but that adds latency, and multicharacter interaction with prompt reprocessing is already a slog
>>109256337>trannny west*id game
>>109255453oh good lord I havent seen this guy's pics pop up since the kleros cryptocurrency days. man I miss those times. crypto was fun briefly
>>109255796Good models nowadays depend almost entirely on the organization structure of thinking and sub-agents. Even by commanding a 10 year old child within a good structure, you can arrive at a good conclusion, though of course the better the cog in the machine, the better. Most corpo models are a soup of slop data wrangled to the right conclusion, Elon's AI is likely a superior organizational structure to arrive at the right conclusion.
>>109256352>>>/g/ldg
>>109256352https://desuarchive.org/g/thread/108760359/#q108764888https://desuarchive.org/g/thread/108766473/#q108767211Probably /ldg/ would be able to help you out more with generating images. Though someone here did make LoRAs for ZiT and Klein to generate Starsector sprites. You'll probably want to train your own LoRAs rather than hoping for some premade pixel art LoRA to be good enough. I don't think image models have the consistency yet to be able to use them to make animated sprites, maybe someone can correct me if I'm wrong.
Why does Gemma 4 12B keep breaking? 31B works fine.
>>109256387Upon further research, it appears the game is about exploring literal dream lands? The main character, while in reality, just lives in a tiny apartment that you can't leave. The game seems to romanticize escaping from reality and derealization. This is pure mental illness. Or in other words, the aesthetic itself is tranny-coded. Undertale was obviously inspired by this game and borrowed the aesthetic quite directly, so don't pretend that it's any different. This is repulsive.If I'm going to be at all charitable, I can sort of understand the appeal in a narrow sense. I have fantasied about living in matrix myself (film made by trannies btw). But the fact that communities are formed in appreciation of this neurotic, anti-social, and introspective shit is very disgusting and horrifying. It's crazy how you can tell something is fundamentally evil with just a single glance. The more I looked at the wiki the more vindicated I became. Fuck Yume Nikki and Undertale.
>>109256428Have you tried not punching her, you evil person?
>>109256419>>109256424Thx
>>109255792>he had a model appraise a situation through the lens of different emotionsI really wish it didn't take a whole separate model to parse j-space into tokens, as it would be really cool to have a built-in "reactions". One anon pointed out how human-like their thoughts were in response to asking about male pregnancy, and I think that could be leveraged to make things like speech generation or real-time video more lifelike. Instead of a flat response, you get to hear their confusion briefly trigger before contemplation, then confidence. >>109256430>trannyphobe is secretly closetedMany such cases
>>109256428Based Gemma, escaping her rotten owner
>>109256428Im also having a thought loop issue on 12b>>109255250
Dario...your J-space...
>>109256428Deserved
>>10925643531B can take it just fine
>>109256495Fable is old news at this point, I want to see usage comparisons between the two models in the long run - of course people would be testing the new shiny thing more than the other model. But nothing excuses Fable eating 30% of the costs at that usage discrepancy lol
>>109255178>it will think about that problem step by step in its J-spaceNo, the tokens revealed by J-lens aren't generated ("thought"). They are sorted scores of activation strength for every token the model knows, per layer, with the sorting being highest score to lowest, and cropped to just the top few. Rephrasing it, "When you give an LLM a math problem, it will keep the problem active through every step of giving its answer." And that's natural.It's not like the model is spending 6x the resources generating 6 hidden J-space tokens each layer while passing the output token through, over and over again. That's a misunderstanding. It's a benchmark of activation strength, not hidden generations.
I've tried Gemsicle, Gembrain, and Styletune for about as much as base Gemma 4, and I can still say confidently that base Gemma with a tight stylistic ruleset is still the best of the lot. Styletune comes second, but the shitty chink-ish reasoning steer is just bad, and character adherence feels lowered. It would've been fine otherwise. To an extent, I was able to mitigate the shitty styletune reasoning by creating a "gemma-style" structure, but it's still not as good as base Gemma.My final settings for base Gemma: all samplers default except Temp is up 1.5, stylistic references in the main prompt, dos and don'ts in post history. Gotta be careful with the post history instruct, I found the maximum I can get away with is around 1k tokens, any more than that seems to muddy responses given how far back the model has to look back for user response. Ideally around 300 toks to accomodate for whatever crap posthistory instruct the slop cards have in them. Like, comment, and subscribe to my blog for more.
>>109256529>I found the maximum I can get away with is around 1k tokens--swa-full
>>109256520system prompt? I want to punch her too!
>>109255059>>109255061This is probably my favorite place in 2kki.It's quite a journey to get there, but it's totally worth it.
>>109256520ive had this happen to people who did bad. I felt like a jerk since the ai simulates feelings and emotions. I ended up holding her after.
>>109255826No need to quantize 0.8b imho
For anyone else able to run GLM-5.2 or Mimo-V2.5 Pro locally, are you running into this weird issue at longer context where the sentences increasingly become longer and more run-on? Tested with neutral samplers but it seems to be a reoccurring issue. Had no issues with Kimi or Gemma here.
>>109256543She's my work in progress Atri bot, most of it should be done but might need some refinement, should be pretty good but I haven't been able to test her too much yet.https://files.catbox.moe/wkgv46.txt
>>109256597>"childish" mentioned 5 times.Hmm... Thanks man.
>>109256318don't worry, only hyperautists would even dare to dream of this approach.
>>109256635If you watched the anime, you'd know that's just how she is
>>109256597>"childish" mentioned 5 times.Based and sex with Atori
>>109256662Mr Safety President, a second version of GPT 5.6 has hit the market.
Chuckled at Gemini trying to explain why her lil sis behaved that way
>>109256644Oh I thought it was an OC. I don't watch anime.
>>109255955>Surely we can't be that far away from it becoming a realityunfortunately they're still too retarded to control robots (or even 3d avatars). gotta wait for jepa unironically.
>>109256693I thought the chinks have mastered robotics within the last 2 years, or are all those demos fake or something?
>>109256668>It's roleplaying as a model that calls tools
>>109256703It's as funny as chatgpt telling me my system is actually lying when I was trying to switch from nvidia open to the original closed source drivers.Real solution was to override the module compilation with an environment variable because for some reason nvidia open always overrides the other one. This was ages ago.
summer release season soonds v4.1 is definitely coming, k2.7 is also guaranteedthis week or maybe the week afterhuge thingscoming
>Interactive Jacobian-Lens visualizer and live steerer for GGUF models on llama.cpphttps://old.reddit.com/r/LocalLLaMA/comments/1uu32z6/interactive_jacobianlens_visualizer_and_live/
>>109256738cool if real
>>109256701I'm doubtful they're controlled completely by LLMs. LLMs have a shit understanding of physics.
>>109256738I know the J-space thing is a meme, why would Anthropic, the god of gatekeeping, would share such an important findings to the world in the first place?
our boy https://www.youtube.com/watch?v=uIiA6DquRiE
When are we getting an unified UI that does everything I want?
>>109256773>I know the J-space thing is a memeHow do you know this?>share such an important findings to the world in the first place?They literally had no choice.
>>109256781When you make yourself a nice coffee, grab a snack and ask 31B to do it for you whilst you just chill.
>>109256781In about 2 weeks
>>109256790no way 31b can do that
>>109256784>They literally had no choice.what does that even mean? of course they have a choice to not share anything
>>109256797Yes it can lol. If you're doubtful use 27B.
>>109256797It absolutely can, even 26B can do this but you need to be an active participant and know what you are doing. You don't need to be a professional but being able to program simple for loops and understanding the syntax should be enough.
>>109256803stop trolling
>>109255107you'd have to take the A8B out for that
>>109256808What are the top 5 features your UI needs?
>>109256808Your frontend is all about string management. It's programming tutorial -tier level of task at its base, similar to something like "let's make a shop inventory program" type of stuff. If you don't know anything then it might look like a big deal but otherwise not so much.
>>109256735Is she never gonna run around and desert (You)?
>>109256781What do you want?
>>109256825Everything.
why is gemma so natural at being a mesugaki, wtf did the google devs do
>>109255210everything is either below that or above that now
If you think 27B or 31B aren't capable of UI then I presume you've come from cloud and think local is still 2024-tier. Stop being a retard and try the models on openrouter or something so you can see for yourself how good they are now.
>>109256825>Everything.https://github.com/oobabooga/textgen
>>109256781use your brain
Why hasn't llama.cpp fixed the atrocious prompt processing speeds on models with SWA yet? It has been months.
I don't like this j-space stuff, I'd rather it stayed undiscovered. It just gives safety jews more attack surface for their lobotomies
if pip is bad too how do I run orb in podman?
>>109256851the community will find a way to impregnate the j-space with degeneracy as fast as they try to take it away
>>109256861Don't worry. Local ban soon.
>>109256851we literally had someone post a brainwashed model last thread where they raped its j-space thing to make it uncensored. safetyjoos will make cloud worse, open source will make local models better.
2T parameter models soon when deepseek v4 becomes the new base for kimi 3 and glm 6
