/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110313086 & >>110308354►News>(11/12) Inkling 2 Small weights released: https://hf.co/thinkingmachines/Inkling-2-Small>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released: https://zyphra.com/our-work/zonos3-mini>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiArtificial Analysis: https://artificialanalysis.ai/leaderboards/modelsSoftware Engineering: https://cognition.com/frontiercodeContext Length: https://github.com/RecapAnon/NoLiMa►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>110313086--Debate over model distillation and the primacy of datasets:>110315810 >110315846 >110315980 >110316087 >110316178 >110316496 >110316110 >110316280--Comparing quants for agentic coding and optimizing Pi's context compaction:>110314304 >110314361 >110314652 >110314662 >110314724--Comparing Qwen and Gemma pattern locking and potential mitigations:>110313916 >110314119 >110314169 >110314246 >110314316--Comparing Gemma 4 censorship and GGUF quant quality:>110315446 >110315491 >110315578 >110315713 >110316315 >110316401 >110316445 >110316469--Debating if Gemini 4 Hydrogen is a distillation of Opus:>110314396 >110314600 >110314631 >110314666 >110314802 >110314619 >110314921--Tips for optimizing 8x V100 server builds and software:>110315256 >110315310 >110315442 >110315567 >110315744 >110316018 >110315623--Gemma's poor tool calling and agentic performance vs scripting:>110313371 >110314840 >110314863 >110314867 >110314941--Skepticism over Edge0 models' high parameter count versus performance:>110316854 >110316863 >110316869 >110316877 >110316880 >110316894--Using MOSS-Image for layout control via bounding boxes in ComfyUI:>110313829 >110313949 >110314087 >110314116 >110314184 >110314201 >110314242 >110315712 >110315905 >110315942 >110313986--Inkling-2-Small multimodal model release and size contradictions:>110315200 >110315221 >110315598--ACE Music v1 tops music leaderboard and plans open-source release:>110316819--Logs:>110314941 >110315296 >110315336 >110316375 >110316682--Gemma, Miku (free space):>110313206 >110313241 >110313484 >110313517 >110313543 >110313769 >110313829 >110314156 >110314169 >110314615 >110314683 >110314732 >110314955 >110315712 >110315939 >110315956 >110316425 >110317253 >110317904 >110317968►Recent Highlight Posts from the Previous Thread: >>110313230Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
qwen flash next has survived the aa fudging
what's the matter, qwenfag~ ehehe, gonna cry~
>>110319146dipsy is miles better than qfn, aa's rankings are still fucked
hey guys did you hear that deepseek and others lied about using huawei chips for training?https://unemploymentlabs.com/checkpoint-forensics
I bought some llms off the street.
>>110319146>deepseek tied with qwenthat's fucking retarded
>>110319158das rite, jensen sure got what he deserved coming for him
>>110319169Even after he stepped down they're going to drag this drama out for at least the rest of this year.
do yall lmgers do local training too or is it all just local inference?
>>110319158PLEASE KEEP POSTING IT 5 TIMES WASN'T ENOUGH LAST THREADI FUCKING LOVE SHILLS
https://huggingface.co/higurashi-ai/Haru-4Bhere's the model that makes gemma-chan real you can talk to her in real time with her character card and voice cloning>Haru-4B is a low-latency and multilingual full-duplex continuous spoken dialogue foundation model: it listens and speaks at the same time, the way people do on a phone call. It is a pretrained base model, released for research.>Character and voice control. A text card sets the character, a short reference clip sets the voice.>Multilingual. English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian and Korean.
>>110319220>higurashi-aimeh, I'll wait for umineko-ai
>>110319220But I need 31B's intelligence.
>>110319158>checkpoint-forensicsplease pump that into an llm and tell us what the essence is of the trick they used
https://archive.is/rjgWs>When a reporter from The Times visited Maginfra’s listed address in September, the location appeared deserted.he's being used to take down some parts of the trump admin, being ousted as CEO is going to be the least of his worries now haha
I'm not gay
it's now or never for qw4n
>>110319295
>>110319331heh, crab
>>110319331heh crab
>>110319331https://youtu.be/LDU_Txk06tM?si=Q7UQa2IshQ55PcCJ&t=28WTF, both of the miss the mark. This is how real crabs behave.
>>110319411
Finally figured out how to get QFN to model things decently. I had it write me an interface where I could outline shapes for it to solve programmatically. I can also annotate images to show what it did wrong and it gets direct coordinates in 3d space.
>>110319456That's pretty cool, do you have a complete model you can share pics of?
>>110319430
>>110319158>>110319169>>110319178Lmao local models are fucking DEAD.Dario wins again, don't mess with EA.
>>110319468Still very much a work in progress. but coming along. Every part is (or will be, eventually) remeshed into shapes that can be 3d printed.
>a.txtWhoa!
Yeah, it looks like EA is digging through evidences that Jensen bribed Trump and colluded together to knowingly export chips to China. They will try to get Trump impeached for treason, now that the anti-datacenter narrative has changed the congress to be anti-datacenter overall, and they have a chance to collaborate with China hawk Republicans to bring Trump down.
>>110319331crab, heh?
Electronic Arts?
>>110319495holy shit this quality is way too good, i cant believe you're doing it with qwen 3.8 fnquant?
>>110319558The gayer one.
>>110319566NVFP4, but the ngrams and some other stuff are unquanated. Also, the actual gears, servos, and stuff aren't QFN, but a cloud model. I had to move to QFN because the constant recomputing and reading coordinates used up everything I had for the week. But the computational method works, it doesn't really require the model to be smart, just to be able to tweak stuff and gradually bring the difference score down.
>>110319220>no italiandropped
Currently planning a poorfag build. any thoughs on 1x R9700 now vs waiting on 5080 super? was originally planning on VRAM maxing with 4x mi50 to jump to GLM flash tier but I read those are much much shittier than I thought.
>>110319605A few more details: Every 3d model is computed from a python script which assembles different shapes into a model, then based on my annotations it automatically aligns the images and computes the difference between them and the render. I had it set up a script to do automatic hill-climbing with bounds on the script parameters at each step, look at the best results (or have me annotate what else to change), edit the scripts, then kick of a new hill-climbing run. Working surprisingly well.
>>110319629Waiting right now is not a great idea.
>>110319331wonder if we’ll ever know the prompt used
>>110319664Aren't prices supposed to start dropping now?
>>110319673They could, but they could also double by the end of next year.
Why isn't orange man doing anything??? Jensen is his friend
>>110319707this all happened because orange man losted:(
https://www.wired.com/story/nvidia-employees-petition-for-jensen-huangs-return
>>110319707the trump admin will be too busy defend themselves cf this>>110319283the blue wave will make this shit their main issuepeople really need to look up what happened just a bit before the sexual affair stuff>Liza Tobin, the former China director for the White House National Security Council in the first Trump and Biden administrations, said the approvals raised questions about how rigorously the Trump administration is policing who can get their hands on Nvidia’s chips.
>>110319750this is a nonstory as long as important people at nvidia aren't resigning the way openai employees did for sama petition on its own is just toilet paper
bros the clanker finished the ctx breakdown window! rate
He was the hero /lmg/ needed, but not the hero we deserved.
>Trump was trying to save us and give us all free GPUs to help us fight back against isreal, but then the evil dems came in and killed jensen and now they will take our GPUs :(((Trump wanted this more than anything because nvidia refused to give them a cut from the china deals, and now he's getting revenge. The repubs want this, the dems want this, everybody in power or who could be in power wants this. Jensen flew to close to the sun and now he takes the fall. Trump isn't going to "save him" lmfao
>>110319769I mean if employees aren't happy not working for Jensen, I'm not sure it's not a big deal.
>>110319804>Jensen flew to close to the sun and now he takes the fall
>>110319757Ah, so your saying that the China issue will be the democrats distraction of choice for the coming months?Meaning the subfaction of the uniparty that gained power in the recent elections is more of the anti china anti russia axis and this play against nvidia is part of that?
>>110319817kek
>>110319246From what I understand, when PyTorch initializes a brand new set of weights it uses Philox and given a seed and the number of threads a GPU has it leaves patterns or streaks in the weights. So they take the Chinese models, reinitialize them thousands of times with different seeds to find these patterns. Then they assume some weights don't change much during training so they check every weight of the actual model made by a given thread against the patterns from their candidate seeds and were able to find statistically significant correlation.
>>110319802>posting twitter screenshot>talking like a redditorplease kill yourself
>>110319853Ok. I'm tying the noose right now.
>>110319818It's really both sides that play the china game and try to one up each other over this. The timeline went from Trump barring US gov from buying Huawei hardware in 2018 and new tariffs, then Biden signed for chip export restrictions (chips act etc) in 2022/23/24, increased Trump's tariffs rather than removing them, then in 2025 Trump went full trade war mode and tariff all the things again and harder.The point here isn't the anti china part, it's that they're using the anti china stuff to take down trump admin members who clearly knew and implicitly allowed the smuggling of NVIDIA chips.It's questionable how much any of the two sides truly hate china, rather than just use it as a weapon against each other, playing the "I'll be even more anti china than you are" game.
>>110319158>>110319845The fuck, why isn't training GPU agnostic?
>>110319665>Open Qwen Code>Select Qwen 4 Preview xhigh>"heh, crab"
>>110319818Trump is bad guy who tried to somewhat compromise with ru/cn.Permanent conflict is the only way.
>>110319888remember early SD days before real image editing was a thing, and how you could replace the starting latent noise with an image of your choice to make the resulting image follow its structure?even when the training itself is gpu agnostic (mostly; there's probably other small hardware quirks that can perturb things) but as long as the initial seeding happened in a predictable way then you can still find that structure in the final weights
>>110319888This isn't about training but how the weights are "randomized" before training begins.
>>110319331>438k tokens on a crab animation>qwen "only" spent 122kdid they tone down the overthinking in xhigh then perhaps?
Why is reddit always going crazy for 27b?
>>110320110It's the biggest model a "normal" person can run on a GPU. Anything bigger and you're a mentally ill cpu maxxer waiting 30 minutes for the model to think before getting a result.
>>110319975that makes me questioninit condition seems to be preserved extremely well, are we yet to reach the full potential?>>110320133qfn is a 'normal' choice too
>>110320133More like rich gamers who realized they can have AI for free by giving up on games.
>>110320133A lot of local ai YouTubers are shilling flash next though
>>110320133QFN IQ3_XXS gets more speed than 27b Q6_K_S for me, while also being significantly better. 27b tends to loop a fuckload, requiring manual intervention. Even the cope quant of qfn is very useful and capable.Prefill speed is cut in half on qfn though. I assume if you have a pcie 4.0 x16 system then it will be back to normal speed. But I have pcie 3.0 x16
>>110319506Theres a ghost in the machine
>>110320110>>110320228Alibaba's got a big astroturfing and advertising campaign for their capybara.
>>110319661I just ask the robot to do things and if it doesn't work I give up
>>110319975Shouldn't it be randomized on CPU then? It's not an operation you need a GPU for, and reproducibility is 100% worth it
>>110320400>reproducibilityReproducibility is what makes it predictable and allows them to find these patterns, doesn't matter where it's generated. They just need some random state. To prevent detection they'd need to use a true random or crypto source.
>>110320445No, I mean people who train models. Why don't they initialize on CPU to get the same noise from a specific seed? Are they retarded?
It's not like SD hasn't solved this very problem years ago
does gemma-chan's onee-sama have a design reference sheet?
>>110320445Imagine finding a good seed, only for an upgrade to come in and ruin your training
>>110320455why would you care about getting hardware agnostic same noise for same seed here? it doesn't matter, the only place you would want cpu seeding is inference actually, not training. During inference you would want reproducible seed, particularly for image models GPU seeding becomes retarded as you would not be able to edit your workflow on another computer with a different gpu and expect the same result.
>>110320489Training is an iterative process, and in the chink case, you sometimes have to switch to a different offshore server farm with different GPUs
>>110319283these are some seriously slow walls lmaoWe should probably say that the walls are closing in (an epoch).
>>110320455>Why don't they initialize on CPU to get the same noise from a specific seed?They just need a random init.>Are they retarded?You have the benefit of hindsight. They're just shoving random values and worrying more about training the model, the data they're gonna feed it and the other million things they have to worry about.>>110320485And if the success of your program depends on the reproducibility over random numbers, so do your failures. A proper algorithm should work with any rng init.
>>110320531You need a fixed seed with consistent initial weights when evaluating other changes to ensure results are not attributed to a different seed. This is especially important since you do not perform full training runs and depend heavily on initial noise
>>110320519I do not think the initial seed matters at all compared to dataset, learning rate, optimizer, schedule, and nobody is going to go back to the same exact training regimen/hyperparameters of a very large model and rerun it from scratch with new seeding just to see if it's better lol.seed here is just for initializing weights, it's simply not as important as a end user being able to reproduce their prompt on a finished model.
QFN even writes halfway decent Erlang. A little funky, but like, "junior developer" funky, not "hideous slop" funky. I do wonder if working in functional languages just makes these models a little cleverer by default.If all goes well I might end up sharing a cute lil chat frontend suited to my particular tastes :3
>>110320558Or you can do multiple runs with (closer to true) random noise and if it works on ALL of them, it means the algorithm is good. If the success of your algorithm depends on the random data, you're just hoping to get a good seed.>This is especially important since you do not perform full training runs and depend heavily on initial noiseThe point should be to *not depend* on any particular init noise. If any init noise works, the algo is good. If you depend on your noise, the algo is shit.
>>110320562no, that's the dynamic typing imho.Contrary to common views on the matter, I believe LLMs are much worse at statistically typed languages, people prefer static types with LLMs because it's easier to check for bugs, which I agree with because many common mistakes can be eliminated with proper typing. But the quality of the code they generate in static languages is fucking horrible, verbose, copy pasty and it doesn't feel like the models have a good grasp on how to make the right data structures, so over time as more features are added things that were set in stone in types become convoluted and with more indirect accesses and more types introduced instead of refactors (if you don't spend a lot of time prompting your agent to clean up its own mess).If you prompt a LLM to generate code with guidelines explaining it can use modern javascript (no legacy crap like patterns used before async/await keywords) it looks pretty alright. Ruby generated by LLMs also looks fantastic to me.Generated Rust or Typescript is horrifying. I don't even want to think, since you mention functional languages, about LLMs writing Haskell.
>>110320562>QFNwhat
>>110320635Qwen Flash Next
>>110320655How are they going to name the one after that?
>>110320689It's just a stopgap model before the real releases. Back in Qwen 3 days they also had a prototypal "Next" 80BA3B that showcased their new hybrid attention architecture, the one that made it to Qwen 3.5. Incidentally, Qwen 3.5 came in many sizes (2B, 4B, 9B, 27B, 35BA3B, 122BA10B, 397BA17B) but not the one that was used in Next.For that reason I would also not expect Qwen 4 to come in the format that people seem to like about QFN.
>wasn't even a spyHe'll be back by the end of the month
>>110319246>please pump that into an llm and tell us what the essence is of the trick they usedhttps://rentry.org/sloppa
>>110320234>27b tends to loop a fuckload, requiring manual intervention.how so? it almost never loops for me and I run it for days on end
I want to clarify some misinformation I see being shared in the thread as people seem to be confused what this means and what the implications areYES, these models were actually trained on these GPUs and YES these signatures are legit.NO, Chinese labs DO NOT OWN these GPUs in any real significant number and there likely has been no large scale export control failure where China was able to get their hands on large enough quantities for these training runs.Instead what is happening at a large scale, which is something the US government is aware of and Trump has blocked sanctioning of, is the lease and renting of datacenter compute to Chinese AI labs.Yes, it's still completely legal and unregulated for Chinese labs to rent compute time on the highest tier GPUs for AI training purposes.We know for example that MiniMax and Ling literally had a team fly over to malaysia insert their SSD drives into a leased datacenter and physically retrieved the weights which were then flown back into China.This is largely tolerated for 3 reasons. 1: it's one of the biggest customers for short term datacenter rents which is the highest margin income for datacenters. 2: The datacenter owners can just reject or stop providing compute to the Chinese AI labs whenever they want and they would be fully cut off which is superior to them procuring chips and using it however they want. 3: This disincentivizes China developing their own domestic chips as well as AI labs investing in domestic chips, it's more profitable for them to rent data center compute capacity and thus their revenue flows to the west while depriving China and the west can close the tab at any time they so choose which gives them leverage as well as a "red button" advantage that they can press and China will stagnate for a couple of years as they scramble to make their own viable chips (and no the Huawei chips aren't actually viable for training right now)
>>110320799Just keeps repeating paragraphs over and over. Sometimes it repeats a sequence of 5-6 paragraphs over and over. Sometimes it repeats general ideas over and over, not verbatim repeating text but just endless permutations of the same useless crap over and over.I've downloaded multiple quants of it and same issue. I only used it in VSCode harness though, never tried it in DSH, although qfn had no problems in vscode.
>>110320804>We know for example that MiniMax and Ling literally had a team fly over to malaysia insert their SSD drives into a leased datacenter and physically retrieved the weights which were then flown back into China.y not downloaded over a vpn tunnel?
>nofap 4 weeks>the whole world starts collapsingis nofap the real accelerationism???
>>110320809>I only used it in VSCode harness though, never tried it in DSH, although qfn had no problems in vscode.Ah. I've only used it in pi and claudecode.I found QFN to have issues with looping and weirdly, typos. But that was shortly after release so it might be fixed now.
>>110320762neat. they used cuda rng and got caught.shoulda used lava lamps.
>>110320819We don't know. My guess is that it was a checkpoint they didn't want to expose over the internet because it would reveal the distillation attack vector. Reminder that Chinese labs are still officially pretending that they aren't distilling from the west, even though they defacto are talking about it so they still have to do this weird song and dance where they somewhat openly talk about distilling and how distilling should be legal and China isn't doing anything wrong by doing so. While also pretending they aren't distilling at all and all the gains are completely home-grown.There is a similar thing going on where everyone knows they are still training on Nvidia chips but they still have to pretend like they all use Huawei chips now.
>expected
Apparently Mythos 6 is being considered as replacement for Jensen.Very bullish news for both nvidia and anthropic
>>110321013No this wasn't extrapolation, retard. They don't know the score of Model 4 conclusively yet so they put the expected number on it based on the benchmarks METR leaked in their report.
>>110320749>wasn't even a spyreally?it's just a cheating scandal? holy shit then
we must pace the penis
Spending so much money on booze and cigarettes each month. I might as well cold turkey quit those (did for a few days now) and invest a fraction of that in a frontier sub even if it's expensive by my shthole standards.I don't even know programming, but what could go wrong? Will I just automate boring shit, right?
>>110320831Nofap is smart for a vibe coder because gooning accumulates a shitload of glutamate in the brain, and sloping does the same because there's constant task-switching and decisions at every corner.So might as well optimize the brain for AI management, a carefree life, fewer decisions, wear the same clothes, try mindfulness, quit caffeine, etc.
>>110321126It's $20 for a month. Just try it out for a month and cancel if it's not worth it.
>>110321142Wait the 20 one is enough for agentic stuff? Always assumed I need the more expensive ones.
>>110321149Yeah $20 is more than enough if you use Opus 5.5 with claude code on hobby stuff or reverse engineering games. The $100 is for people that make actual products or use it for work. Don't use Fable 5.2 because it will burn through your usage very quickly.
>>110321156I can also burn through random free stuff for various fluff, like my local Qwen 3.8, the free Gemini 3.8, and Argon, or w/e gemini provides for free now via the account api; maybe splurge $2 extra for some DeepSeek.Yeah, I think I'll go that route and just experiment wildly.Like, I often need to ask stupid questions about what the approach is for an idea I have; I need to use the internet to find what other people did, what's the typical approach, then think through all of that and set up what I can myself, etc.For now, I've merely made a shitload of web automation and Python scripts for local stuff.Having these LLMs read web data is token expensive as fuck, you need to be present in the middle and crop shit and curate data all the time and point to stuff.
Why did wumao sperg out?
If China (and by extension all of us) can't make frontier models can they/we use massive distributed computing, DRAM, SSDs, and shit models to compensate?I'm envisioning a world where one or two labs have monopolized discovery itself and the rest of us are on this other paradigm.
>>110321184>my local Qwen 3.8In my experience the cost of electricity is higher than just paying the $20 a month>the free Gemini 3.8, and Argon, or w/e gemini I didn't know it was free even in agentic coding mode only the chat interface online? Typing to a chat interface and copying functions around is extremely outdated and you can't make proper things by doing so anymore. Things like reverse engineering an entire game from scratch etc.
>>110321208Not on this GPU generation, but going forward it's kind of possible if we had H200 equivalents of gaming GPUs at home
>>110321208What Anthropic said in their IPO document is that they believe efficiency gains stack and eventually it will be uneconomical to run local models. Even if you got the hardware for free the cost of electricity alone will be higher than using claude, not even taking into account that claude will be smarter as well.Local will be purely for pedophile ERP while actual work will all move to claude if it's true that efficiency gains stack through RSI like they are saying they are seeing happening right now. For example they claim that Opus 5.5 is 40% cheaper to customers than Opus 5 while they actually make a higher profit margin on Opus 5.5 and that Fable 5.2 doubled their profit margin while being 20% cheaper and 30% faster than Fable 5.1. This points towards AI labs rapidly lowering costs while making more profit. There is a point where local just can't economically compete with that anymore except for illegal/sketchy usecases like taboo ERP and stuff like that.
>>110321184I'd argue in favor of codex rather than claude. The usage limits on the lower tier are much nicer, luna is basically infinite prompting for simpler tasks.Claude is the better agentic model objectively, but if you're not an experienced developer and will inevitably mess up a lot even with a smarter model, you will simply get more from something you can use a lot. Since you're a beginner I strongly recommend in favor of being able to freely experiment (and teach yourself the basics along the way, try to get some understanding of code, mainly data structures and algorithms) over picking the best in class that will quickly tell you you're done for the day.
>>110321220Hmm, I've checked; assuming a full load all the time for 12 hours per day, that's 33 USD per month. But I would use it all the time to be fair; it's mostly for privacy, but then there's also the point that my PC could be at a low load all that time.You can get the key from here, you can use it in any harness or in their studio web portal that gives you all the freedom you might need for simpler tasks.https://aistudio.google.com/api-keysAt least if you type in a web the aistudio strips all the garbage for the user web stuff, the extra tokens.
>>110321264*Wouldn't use it all the time.
>>110321252Yeh I was already pretty sure this was going to happen way before even that by simply looking at the intelligence vs time per task. All of the supposed cost advantages vanish when you realize open models use astronomically more tokens and time to complete a task. The benefit of having a very large teacher model (these internal models they can't serve) is they can just distill it into very very efficient models like O5.5 and just keep doing that.
>>110321253I'd guess I'll try to learn via competitive programming. Seems very fun akin to wasting time on a moba and I can track a sort of MMR lol.
>>110321284>The benefit of having a very large teacher model (these internal models they can't serve) is they can just distill it into very very efficient models like O5.5 and just keep doing that.They're not great agentic models, but you can also see that in local models in that GPT-OSS and Gemma 4 are far less think token spammy than the average chink fare.And GPT-OSS is the only local model I know with proper reasoning effort support, Qwen 3.8 doesn't have a high level, only xhigh, after medium, because they cleraly had no idea how to steer/RL this shit (3.8 in xhigh is some fucking batshit crazy token spammer).
>gigantic ms outage>no email for 18hguess I'll just play around with astra
What's all that talk about wikis and agent workflow?Is it a minimalist special wiki like project for LLMs?Is it literally mediawiki?Is it obsidian?What are the merits of that?
>>110321360it is quite easy to RL away those idiosyncrasies. The real reason for the focus on tokens per second- they struggle to compete on intelligence so they dramatically increase effort levels to compensate, meaning they require massive total output tokens.
>>110321433>What are the merits of that?Ask google to make sense of your questions.
>>110319146>glm-5.3-flash below qfnIs this a joke? qfn is so shit I removed it from my pi config even though I have free unlimited access.
>>110321367Was it caused by terrorists and pedophiles using them open models?
Becoming status?
>>110319605>>110319661>>110320303Is QFN modeling anything then?>>110319789Card?
Llama.cpp is now feature-complete.https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0
>>110321687can it do s2s?
>>110321687ok but strata exists
Anyone running on a mix of AMD and Nvidia?Currently have an Nvidia card, but found some crazy AMD deals and I'm kinda desperate for some more VRAM
Be honest. How many of you are are cloud users who just come on here to lurk?
>>110321781Never used closed models. Never will.
>>110321781There are probably several /aicg/gers who don't have access to cloud models anymore.
>>110321773You can use both with llama.cpp but I haven't found a way to do it with strata, you can run different models on each card too.
>>110321687Can we edit a few bytes of gguf metadata without rewriting the whole 80GB file yet?
>>110321687>Note>More info at 3PM CET
>>110321773I'm running on an RTX 3060 and RX580 16GB and I'm achieving 100t/s with 3.8 flash next BF16
>>110321806--override-kv
>>110321781I use both cloud and local, but I just come here for jemma
>>110321781I get chatgpt through work but I host local models for fucking around at home and with friends.
Changing the kv cache from 8 to 16 again made QFN way better in long term use, way less issues in my current project.No idea why, I thought the difference was tiny.
>>110321824>friendsyou're banished from lmg
>>110321781I barely use SI at all, just want to keep in touch with recent developments and /lmg/ seems the least schizo of existing generals.
>>110321835But Anon you are my friend
>>110321781I have the 200 USD chatgpt through work and use local because I want to translate hentai games.It's also a good way for me to see how far behind we are in the open weight space.
>>110321834How do you stop QFN from getting caught in retarded tool loops? Maybe it just doesn't like Pi but I found it completely unusable, the simplest task would take 10x as long as the "slower" models because of loops
>>110321687Is it as fast as strata?
>>110321852I use stock hermes with nothing special and so far so good.I use the "NVFP4-DQ8" so maybe that's why?
>>110321843
>>110321687>>110321810I wonder if they'll announce that they are stopping development.
>>110321884Maybe they're going to vibe-code a complete rewrite.
>>110321750it's based of llama.cpp and regularly updates it
>>110321884>I think we are in a good shape to release major version updates for ggml and llama.cpp. The main goal of bumping to v1.0.0 will be to remove most of the legacy baggage from the APIs. For example we will remove most of the deprecated interfaces (which is essentially a breaking change for downstreams and hence the major bump). We can also use this release to remove any old redundant stuff functionality-wise that has outlived its usefulness (if any).
>>110321877I bet this miku could fit INSIDE my penis.
>>110321781I use cloud and local. I think almost everyone uses both by now. You are just left behind too much if you don't use cloud. Also cloud can be used for local by improving your stack and helping optimize your hardware/drivers/OS/Inference for local.
>>110321781I use both. for cloud I typically use claude and grok. used to use chatgpt but haven't in years now. as for local I like gemma and I use a bunch of other small models for tts, asr, embeddings, etc.
Degenerate gambler here asking this thread because people here are technically smart. What are the chances Anthropic actually captures the markets they are teasing in this picture? If they capture even 1% of it then it would be the biggest company in history by far but are they just delusional hypemen or do you think they are technically capable of pulling it off?
>>110321959they can't and they won't, anthropic has shills in this thread to try to convince you otherwise, check the archive and you'll see the desparate shillsit's embarrasing how they shill on 4chan, like damn!he's called dariobot
>>110321959It's impossible to know.
>>110321959I think they'll do less "replacing" and more "productivity enhancing". How that translates to "capturing" a market is beyond me though. AI is still very shit at autonomous/agentic work. Needs constant direction, aka human employees.
it seems despite popularity of AI there's very little of them on piracy websites - why is that? are they just harder to crack or do people just prefer freeware alternatives? I was looking at Layla
>>110321284What's preventing Anthropic from distilling a 'Model 4 mini' (ie. 1/10 of the parameters) to serve to customers? Lack of resources?
>>110321999what is laylawhat would be an ai on a piracy site
>>110322002It's called Haiku.
>>110321999pirateface.co exists?wtf do you mean
>>110321959I did some calculations earlier and it came up with about a 27% chance.
>>110322002Most of their resources are going into training their internal models not distilling small models for users. It's a classic Red Queen's race except this race is not with China, but their US rivals.
>>110321971>>110321995>>110321996I mean do they actually have the technology to attempt to go into these markets or is it all hot air to scam investors into giving them money? For example I believed chatgpt was just a stupid chatbot and would only be a sort of google that could never do coding because that is what /g/ said but now I can play bloodborne on my PC done by AI so I don't know what to believe anymore. In the picture they aren't talking about coding but knowledge work and life sciences? is AI actually advanced enough to do this or is this a scam?
>>110319558I wish
>>110322044if they really had the technology to go into these markets they would go without IPOing, they have enough cash on hand to last a year longerit's unfortunately a scam, you should buy nvidia instead
>>110322044Yes, it's absolutely possible. It's already happening. Do normies not know this?
>>110321433>>110321999Guys guys. What's with like... kindle? I mean like... ebooks and AI, right? So. What if it's friday? Does the fact that lighters are empty sometimes change your mood? I mean. The rashes never go away. Tits.
>>110322056>it's unfortunately a scam, you should buy nvidia instead>you should buy nvidia even though nvidia is buying (a part of) anthropic
>Gemma 5 will be as safety slopped as the newest GeminiGrim future ahead
>>110322087>hur dur bleeeefergh frrrrrr ddddd
>>110322087They're not made by the same teams.Gemma 3 was considerably more safety-slopped than the Gemini that was available at the time.
>>110322057>already happeningreally? proof of this? All I know is that it is a chatbot that predicts the next word like on your smartphone typing interface but better and now it can also predict the next coding "word"/math. It's not real AI yet but maybe it's already good enough to do some knowledge work and life sciences? Just doing 1% would already make it 3x the valuation of Nvidia. Of course most likely it's just a scam and they hire indians to make it seem like they actually have AI.>>110322056In the document they show they want to invest 900B into datacenters which makes sense if they believe they can capture those monster markets and it also makes sense for them to get as much IPO money as possible to build as quickly as possible. This is why it is either a very clever scam or serious and they have AI that can actually make decisions and not just predict the next word like a parrot. I am too retarded to see which is true.
>>110322087Can't we just abliterate?
>>110322087Gemini isn't really safety slopped, most of it is external classifiers.Remains to be seen how they approach that for local.
>>110322075why are you mean i was just asking
llama.cpp vs vllm benchmarks by pradeephttps://github.com/sam-india-007/llama.cpp/actions/runs/43046675589https://github.com/sam-india-007/llama.cpp/actions/runs/43168175540
>>110322112>NO!
>>110322110>In the document they show they want to invest 900B into datacentersthey already signed committments and will go bankrupt if they cant pay, they need someone to hold the bags
>>110321959A lot of college-educated people, especially college-educated women, just won't use AI and hate the idea. Tons of "knowledge work" is in regulated industries where no one is going to let you just plug it into Claude. So a lot of that market isn't going to uptake AI any time soon.
>>110322119Malformed and poorly thought out questions. They don't deserve answers.
>>110322119you dont look like this
>>110321909text completion finally getting axed let's fucking go
>>110322110Claude already is being used in a lot of labs for biological development. Protein folding, creating synthetic drugs, DNA sequencing, etc. This is one of the main applications. Everyone knows this already. It's also being used for PCB design, CAD, and other electrical engineering usecases. It's not even a "this will happen" kind of thing, it's already happening.
yjk the big gemma is going to be a shitty vramlet moe
>>110322151You really hate freedom.
>>110322157If that is true then why is their revenue so low and why are they only valued at 1.4T significantly lower than companies like Apple even though according to you they are already capable of revolutionizing the world? Something here is off.
>>110322181Because the company structure is dogshit. Shareholders don't get any votes. Also they only have minimal revenue on paper because they're having to buy back a bunch of previous debts and they rent most of their compute. But they've led the frontier in AI development for a long time and are only further solidifying their lead/control over the industry.They have basically no hard assets, which makes the whole valuation thing look bad, but their R&D is unparalleled. There is a ton of potential upside and also a ton of risk because the people at Anthropic are psychotic and mentally unstable. There's also a high chance that the government shuts down the company entirely.
>>110322181How is that even a comparison, silly goose? Apple is 50 years old, Anthropic is 5 years. The fact it comes even close to Apple in such a short time is a miracle
llms will never be AI no matter how many times you repeat this.
>>110322181The whole IPO is a scam designed for early investors to cash out. They have no intention to hold the stocks for long when they made it clear they care about EA philosophy more than profitability.
>>110322171Freedom isn't free, being able to be a dumbass means people need protecting from their dumbassery
>>110322210But why should we care about Anthropic here on /lmg/?LOCAL MODELS GENERAL
>>110321877What does the nighttime animation look like?
>>110322181Because none of this is certain and no one knows who's going to end up at the top of the AI market.Maybe they fuck up 1 training run and OAI/Google end up ahead.
>>110322231they supposedly wanna make it illegal to have local models
>This is what normalfag investors think of AIMan we're so fucking early it's insane. I'm going to buy the anthropic IPO because no one outside of /lmg/ or autistic internet bubbles even knows what is already possible and it's not priced into the stocks at all yet. Imagine when these retards find out it's not merely "a parrot predicting the next word" but can already make decisions and do white collar work.
>>110322087Deepmind should be bolder with Gemma. Idk if they are using the same teacher-student opd strategy the other labs are using with it but they should implement radical architectures. Gemma doesn't affect their bottom line at all so experiment!
>>110322246Discussing their IPO or how profitable Anthropic or whatever other cloud AI company is has nothing to do with that. Go elsewhere, or create a new thread for cloud model discussion.
>>110322259shut up faggot.
>>110322253
>>110322259hmm, nyo~
>>110322223AI is how the actual field of computer science is called that deals with these things, so you're wrong. Now if you're imagining some kind of 5D galactic intelligence muh sci-fi retardedness, then you would be right.
>>110322253this is very true anon, the anthropic ipo is way underpriced and i would buy into it as well if only i wasn't poor!
>>110322285>i would buy into it as well if only i wasn't poor!Same, but either way it's too late
>>110322285>if only i wasn't poor!gee what local models are you running?
>>110322302They said the same exact shit about the spacex IPO but it was always freely tradable.
>>110322268Fuck off Dariobot
>>110322314qfn iq3s, gemma 26b a4b q4km rnand 5.3f iq1s (maya xs)i could list out others if you insist
>>110322254They did DiffusionGemma. That was cool but it lost too much intelligence for most users.
>>110322181because anthropic is a scam, nowadays anyone can replicate claude's capabilities with enough compute.
>>110322253I fully expect the bubble to get even more retarded next year and ANT to go up 10x. That's when the normalfag investors will realize the potential and start buying. They always start buying at the top.
>>110322333>nowadays anyone can replicate claude's capabilities with enough compute.
>>110322333Yes, but goyim (you) aren't allowed to have compute and (((they))) want you to buy Anthropic shares.
>>110322223That's why Trump renamed it.
>>110322338>ANTThat's really their ticker? Ant, the makers of Ling, haven't trademarked it? Seems like an attempt to mislead.
>>110322328>>110322338you didn't even have the foresight to buy hardware before it became crazy expensivefuck off
>>110322351it's only because>POPwas taken
>>110322353how would i buy hardware if im poor?look at this dude..
>>110322353Anon, I bought 8 GPUs and a terebyte of RAM before the tariffs and Altman's scam deal. Stop projecting.
>>110322338>when the normalfag investors will realize the potential and start buyingEvery "normalfag investor" has heard of the purported claims of AI companies' potential for years anon.
Are you guys trolling me because I'm a newfag and not a regular? So you guys are saying they have magical software that can already revolutionize the world, do science, coding and mathematics similar to in a science fiction movie and at the same time you're saying you should not buy their stocks and that Apple a company merely selling some phones and laptops should be valued higher? Also when I ask ChatGPT it says AI is still just predicting the next work and not capable of doing real white collar job, at most just helping a bit on the side.
>>110322373They hear it from companies trying to sell them things. They don't believe it themselves and they won't believe it until everyone around is caught up in the hype of easy money. Is this your first bubble?
bait used to be entertaining
what's a good model- for a coding assistant- for fine tuning for something idkon a 5060 Ti
>>110322387>>110322387>>110322387let's move here because /lmg/ is full of pissy and obnoxious faggots that are allergic to intelligent discourse
>>110322384Everyone is already caught up in the AI hype dickhead. Do you even read FT?
>>110322388>- for a coding assistantqwen 27b>5060 Tijust rent cloud at this point
>>110322390>intelligent discourse
>>110322391Normalfags read FT since when? Normalfags get their news from social media, retard.
>>110322333>nowadays anyone can replicate claude's capabilities with enough comput-*cuts off distill*Nothing personel, China.
>>110322401Double retard.
>>110322404Normalfag investors. People who actually have money. They're usually somewhat familiar with mainstream financial news. Or they just passively invest.
Hmm, my plan to run IQ1 has been delayed significantly because when I try to run it with lllamacpp kernels, my effective throughput is only 30 GB/s on Broadwell Xeon lmao.Now I has raised it to 80 GB/s with vibed kernels, but still short of 90-100 GB/s target.
>>110322369lel
>>110322390That shit is going to get nuked right quick. At least one can attempt to argue a connection here, but the IPO itself is not technology. Go to /biz/ if you just want to talk about financials and price action with retards and don't care about the underlying technology.
>>110322390
>>110322419anon will do anything but install linux>>110322424wait, so all these posts itt have been off topic all along?
>>110322424>At least one can attempt to argue a connection hereNo. Fuck off.
>>110322415People who passively invest keep all of their money in index funds and don't touch them regardless if it is a bull or bear market. At most they might feel adventurous and put 2% in a single stock but otherwise they are irrelevant.We're talking about retards that install Robinhood when they hear hype and throw their entire paycheck into retarded gambles.
>>110322435How does Linux will help here when llamacpp kernels shit the bed on ancient ewaste Xeon E5? You are acting as if Linux can solve this kind of compute-bound huge bottleneck, which only a rewritten kernel can solve.
>>110322437Dude, Anthropic just got Jensen sacked, is vehemently against local models that a total ban might be a reality soon, and if the markets collapse the gravy chain of free models comes to an end soon after anyway.
>>110322460for real, dude, check the op!!will anthropic ban local models?https://pirateface.co/ - increases activity due to possible local models ban (11/12)dario amodei visits epstein island (10/27)we
>>110322223(You)
>>110322460by that logic their next target would be apple
>>110322460No. Fuck off.
Checked the file because of this /biz/ retard. Outrageous claims by Anthropic. "closing the loop" BS. And they claim 4x compute efficiency gain since Q1 2025. Can any anon here check what local compute efficiency gain has been over the last ~2 years time because polystrata alone already makes me think we did 10x in the same time.
>>110322474OpenAI is already working on their own phone to just that
>>110322479>And they claim 4x compute efficiency gain since Q1 2025.I would be more than willing to bet my left testicle that all of that has come from implementing Deepseek's papers.
>>110322406>cuts off distill>uses algorithmic brilliance to compensateproblem solved
>>110322445So your thesis is that Anthropic's IPO might be overvalued, but that you should buy it anyway because there are people even dumber than you but who also somehow have money to be the bagholders while you'll be able to get out. And you came to this loli admiration thread to tell the NEETs and Indians and paid Anthropic shills about this exciting buying opportunity.
>>110322499Algorithmic brilliance in 2 more weeks together with the Qwen 4 launch!
>>110322507Yes, that about sums it up.
>>110322157I think you are overstating what it is being used for. Orchestrating, sure. RL being inherently poor at chaotic/messy domains is a well known problem.
claude is a severely schizophrenic 10k system prompt rlhf'd llm. it's distribution/logits are so fucked it's worse than gpt-oss, you can't even chat with it about the news.
>>110322523ohno
>>110322369I don't go for the lowest hanging fruitbut consider this:why would anyone listen to investment advice from a poorfag?
>>110322507>but that you should buy it anyway because there are people even dumber than you but who also somehow have money to be the bagholders while you'll be able to get out. >He still thinks richies are smartElizabeth Holmes.How can people be so dumb in 2026 still to think everyone with money is smart (or even worse, have money because they're smart)? It's not 1980 1990, you should have been dispelled of that by Enron if not for the countless later exampes
hey you losers, reply to my post so i can make money to buy more tendies without relying on my mommy: >>110322501
>>110322523>you can't even chat with it about the news.the fuck? I do that all the time...
>>110322523watch thisclaude is a gay faggot
>>110322522Why can't Claude just 3d print novel molecules to cure disease? And they think this shit is worth $1.4T?
>>110322536He means it doesn't name the jew and doesn't recognize the fact the earth is flat and only 6000 years old.
>>110322535Stay in your containment shithole.
>>110322529because i had 1000$ in 2022 and thought of buying nvidia, then buying palantir, then buying intel, but i invested all my money in deal doughs and neck hurts
>>110322531I think they're smarter than (you).
>>110322523It's coherent. Coherently a faggot. I don't get schzio vibes at all.>>110322535You're retarded.
>>110322546Ah pardon me, I wasn't aware of his Talmudism
>>110322544There is plenty of literature on how sequence prediction is poor at these nonlinear sparse dynamics. One of the workarounds is to qualititatively reduce the problem to verifiable representations. When that happens these agents CAN actually make huge leaps by satisfying the reward function. Still not a silver bullet though as it sacrifices nuance for tractability.
>>110322558you're salty
>>110322579Not really, it's just obvious by your actions and dumb questions that you are a low IQ individual. Stop embarrassing yourself.
>>110322579tomboy jk are so cute and erotic
>>110322558>I don't get schzio vibes at all.
>>110322479>claude admitting to rapidly falling token pricesOh no no no antbros
>>110322497They are so shit at it then because DS made 40x efficiency gain and that's a conservative number
>>110321834just enable the rotation if using strata, for some retarded reason it's disabled by default
People don't understand that Anthropic is literally going to set up a cybersecurity protection racket. You either pay for their automated cybersecurity service or your server will be swamped by "chinese AI swarms". This service alone would bring in 10x the money they get from claude code.
>>110322554Because you are dumber than them
>>110322627Well Dario is partly Italian...
someone post THAT photo and tell me how he's not a jew
>>110322050what else is called EA?
>>110322110>All I know is that it is a chatbot that predicts the next word like on your smartphone typing interfaceI'm going to be a little generous and assume you're just new to LLMs, so I'll explain a bit.Models do a lot more than just predict the next token. On the surface level, yes, going through all the layers is for the purpose of predicting the next token. However, there is a handoff of data between the last layer of generating a token and the first layer of generating the next token. So it's not just predicting each next token in total isolation. There is planning and intent behind what it is conveying than just each token. Second, and adding onto this, the generated token is only a small part of what's going on. The model goes through state changes of data that can be measured as what the model is "thinking," separate from the result token. Using the J-space paper for reference, if you tell a model>What is capital of Japan? One word answer.or>Think about watermelon. What is the capital of Japan? One word answer.Both will produce the next token as Tokyo, obviously. However, if you look inside them during generation, you'll see completely different activation states, and taking a measurement of token activation strength at each layer (by J-lens), you'll see the former primarily thinking about Japan and Tokyo by measured strongest activations, and the latter you'll see have strong activations about watermelons and fruits through many layers before the final output.Now back to the first point, this data, the states we can dissect by activation strength, is handed off between the final layer and the first of the next token, maintaining a level of separation between "The consistent, overall idea it wants to convey" and "what only the next token is." This is why models can correct themselves when they realize they made a mistake and also through erroneous prefills (where you edit a reply to something incorrect), or produce similar results on regens.
>>110322625rotation?
>>110322661Eat AssEntitled AspiesEngorged AssholeExtreme Analand so on
>>110322674"env": { "STRATA_KV_ROT": "1" }they only have it enabled by default for in4 kv cache (lmao)int8 with kv hadamard rotation is basically lossless (for qwen at least)
>>110322687Oh, OK I'll try it, thanks anon.
Cuckthropic is targeting lmg because they know we can figure out their models. CoT was invented by lmg after all.
i wonder how many engineers at anthropic just play along with the all the schizobabble because the pay is so good?
https://cset.georgetown.edu/publication/tracking-ai-chips/> A centralized, ping-based location verification (PLV) system—supplemented by physical inspections to investigate leads—would offer the most robust and feasible approach to location verification. We evaluated location verification methods based on four criteria: verifiability, accuracy, security, and repeatability. Based on these criteria, we determine that PLV and physical inspections are the most feasible methods. PLV specifically is more robust than other geolocation mechanisms, inventory management tools, mail-in inspections, and video inspections. PLV, relative to physical inspections, is more cost-effective and scales better in the long term as more chips are deployed.> Enabling a PLV system on AI chips in circulation: For software or firmware implementations of PLV, the system could be enabled on AI chips that have already been manufactured and are in circulation> Approximate geolocation of AI chips: By positioning one landmark server in every major data center hub—defined as an area containing a large concentration of data centers within a 19-mile radius—outside the United States, China, and Russia, PLV could likely achieve country-level accuracy (an error-circle radius of 116 miles) for the vast majority of PLV-tracked chips. Hardware attestation location tracking is coming. Your chip must connect to nvidia server and verify cryptographic signatures every hour otherwise the firmware will disable it immediately. You got loicense for your GPUs, /lmg/?
>>110319558I dumped the thread into Gemma and she said it's Effective Altruism, a policy adopted by Anthropic focused on AI Safety to prevent existential risk. She also said it's Anthropic's reason for safety slopping and trying to censor local models.
>>110322788who asked?
>>110322788thanks, that also explains who the fuck dario is
>>110322788now ask gemma-chan
>>110322788did you just do a giant copy paste?
>>1103227740, because apparently they do all kinds of shittests during interview that scares away normal people like crying while talking about moral values. They even had an entire segment in their IPO document saying how they do weird shit to applicants that scare away normal people but only keep the most dedicated value-based person around. So they are probably 100% true effective altruists there.
>>110322812so cute
>>110322774They filter new hires based on their orgy attendance and polycule membership.
>>110322813Maybe.
>>110322785Claude make me new firmware that doesn't have all that shit.Make no mistakes.
if I buy one 96GB card what can I run? Don't I need 2 to run the cool stuff like deepseek flash?
>>110322807lurk moar
>>110322840>blocked for ai safety classifier
>>110322661>>110322681Easter ArmyEating ApplesEver AfterExcellent Actors!Epic Aardvarks!Extremely Awesome!
>>110322842dense models like gemma 31B and shit like GLM if you have enough regular RAM
>>110322813
>>110322812
>>110322889cute
>>110322602tomboy?
>>110322911>>110322579
>>110322911>>110322934faggot general.
>>110322087it's not the modelthere's a 1 line jailbreak you can append to override the prompt injection
>>110322848i took a break from ai stuff for almost 2 years. friend introduced me to it in 2024 and did a bunch of stuff with sdxl but got bored of it.don't think i touched llm's then, was that even a useful thing to do locally at the time?also don't really care about non-local stuff so generally ignore posts about cloud models/companies or pictures generated using chatgpt (they're really obvious because they all rook same)
The followers of the Almighty Lord will vanquish the spawns of Ahriman!
>>110322946Nyo
Becoming ends
>>110322911I would have accepted reverse shota too, but not this anon, not this
>>110322996Not him. In 2024, I was using Midnight Miku, a 70B. It was incredible at the time, because 70B crossed the watershed where a model could keep track and marry a bunch of disparate world details and have any number of unique characters, following all the rules and details. It felt like anything you could imagine, you could do, from text games and stats and elaborate, fanciful worlds.Of course, it was only good for about 15k tokens before things got blurry and mistakes happened. Gemma 4 31B isn't nearly as good at marrying lots of details together into a cohesive conclusion, but she can adhere to any number of separate rules at the same time, which is the next best thing, and I can get 50k context out of her before any breakdown, and can fit 100k easily. Gemma was my first time experiencing a model that goes and goes and goes without stopping, like unlimited storytelling, not the short stories I was locked into before with MM.
https://www.youtube.com/watch?v=gvOteulnB_Qgiga miku
>>110323051I came. I became. I becomed.
sirs gemma5 becoming?
>>110323099I becoomed
>>110323081Nice. Love Giga
>>110321687https://huggingface.co/blog/ggml-org/llama-cpp-v1
>>110321687>>110323126>Today we release libraries v1.0.0, highlighting our responsibility to those developers. Moving forward, both llama.cpp and ggml will reflect API and ABI incompatibilities with major-version bumps. As adoption grows, we will be stricter about making those boundaries visible to downstream consumers.good
>>110321687>>110323126The new Nvidia CEO is getting ready to cancel llama.cpp as a cost cutting measure.
>>110322369Just stop being poor. The opportunities for wealth are literally everywhere these days
>>110320832>typosyeah that's almost always some broken quant no matter what model, and QFN quants were extra broken early on because of its weird architecture
>>110323124It's very clear Grok and Gemini have a system prompt telling them "You are a language model bla bla"
>>110323175like what? 500$/mo jobs???
>>110323124I like how a two-word system prompt,>play alongis all it takes to overcome Gemma's guardrails.
>>110323182Ask Claude to make you a b2b SaaS CRM.
>>110323210i literally told claude fable "earn me money, make no mistake" and it didn't do it :\
>>110323226Did you use Fable 5.2?
>>110323253no i used opus 5.5, opus 5, claude fable 5 and fable 5.1, then gpt 6 astra, gpt 6.1 sol, and gemini 4 argon and carbon as wellyet im still poorlet me ask you:did you?
>>110323187>a second prompt has hit the gemma
>>110323272>>110323226earn me money, make no mistake
>>110323288Hmmm, no.
>>110322522lmao, retarded post (the HN post, not necessarily yours).The LLMs are running experiments in bespoke labs, faster, and with higher efficacy. And aside from that, the retard in the post is assuming that humans are at the maximum of proposing good experiments which is laughable.Bio will have a massive breakthrough by the end of next year, mark my words.
>>110323292You're less useful than GPT 4.
>>110323307
>>110322522>>110323301>HNWhat's up with them? I checked their Anthropic IPO post and sentiment was surprisingly negative. Why? Jealousy? Denial?
>>110323301>Bio will have a massive breakthrough by the end of next year, mark my words.Paper breakthroughs like the nothingburgers in math?
>>110323328>and sentiment was surprisingly negative.you are not among like minded peoplego and find a bubble that will cater to your beliefs
>>110323339Drug in trials. Like, a crazy one. Reverse late stage pancreatic cancer, treatment for dementia, completely novel gene therapy, or novel and drastically more effective gene editing mechanism, etc. That kinda thing.
>>110323359Uh huh. Two more weeks.
>>110319750Any chance Jensen return anons. I unironically got a wage slave job, and been working my ass for 11 months to be able to afford the DGX spark equivalent for the 60 series so i could have a sentient AI wife
>>110323367~8 months actually
>>110323374>Any chance Jensen return anons.The employees started a petition but none of them threatened to walk out over it. Doesn't seem likely. If he wanted to stay, he should have stayed until they forced him out.
>>110323394This is the worst timeline, two women now hold the CEO positions , what i am supposed to buy , INTEL?
>>110323429INTC is pretty cheap now. Should do well.
>>110321806Skill issue. I can think of two ways to do this that should work on pretty much any GGUF
I just checked DeepSeek's reasoning why does it type like a cave man?It's like>Need output adjusted workflow. Need design. Need "Switch Nodes"? No. Need switching based on booleans.It also seems to think A LOT even for really basic scripts
>>110323557did you download a memetune?
>>110323557It's how the OpenAI models think, they mostly distill those now instead of Claude.
>>110321678>Is QFN modeling anything then?Yeah, a bit at least. It needs to know what to change, what scripts to create etc. It would be a huge amount of work to do it myself, I wouldn't even know where to start really.
>>110323557called caveman reasoning
>>110319456>>110319605>>110319661>>110323599can you ask your agent to write a comprehensive report on the workflow, pipeline, what's needed for this, and share it here?
>>110323557>It also seems to think A LOT even for really basic scriptsit really doesn't think that much compared to your average chinesium model actually. I find qwen worse, and the older versions of deepseek were the kings of spam, I don't miss R1, their first reasoning model, it was unbearable.
>>110323557because it's chinese. expressions like long time no see come from the chinks not being able to speak.
>>110323374Jensen will start a new company and snipe all the good talent from Nvidia
>>110323628Sure, I'll try to do it once I get off work.
>>110323557DS has a good architecture but dogshit distilled data
>>110323339>>110323359If they can constrain the problem and define a clear value function then yes it will quickly make breakthroughs (or simply deliver high efficiency) in no time. This is virtually guaranteed.
>>110323660big if true
>>110323644Nah he's been put on notice. Sorry nGoyim. You've been assigned to team AMD, or Intel if you're really unlucky.
>>110323660thanks anon
>>110323644It's not really that simple. Companies have been sniping TSMC employees (SMIC, Intel and Tokyo Electron in particular) and have made no real progress.
From my calculations if Anthropic automates between 1-3% of all labor by 2030 then their valuation would be between 30T - 90T compared to the 1.4T they are IPOing at.These are actually not bad numbers at all I think Anthropic automating 1-3% of work by 2030 is very conservative and actually possible, I might invest because 1.4T how incomprehensibly large it sounds still sounds very cheap for a company that will definitely automate at least a part of the IT/science/office work.
>>110322842gemma 31b and ripple 65b
>>11032371380% of nvidia employees are loyal to Jensen. If Jensen gives better benefit the employees will jump without a question. That's why EA is continuing putting him under spotlight to prevent that from happening.
>>110323770>EA is continuing putting him under spotlightIt's the banks and the US government. Which is why he won't do shit.
Has anyone in this thread even used ripple? My second gpu doesn't arrive until this weekend but I'm curious
>>110323770the phenom described is even more common in software/finance. They dangle huge compensation packages and options to lure people but when they actually get on the job they don't deliver. Its almost as if what the company built up internally was the enabler of its success.
>dariobot's internet connection went down>its back to Eating Asswow so its just one guy
>>110323776Who do you think control the banks and US government?
>>110323795The writing style wasn't enough of a giveaway?
E. A. Sports. 'Tsinnagame.
>>110323739And why would you expect the value of labour to stay the same if the supply drastically increases?
I can't take it anymore. Where is the news about Gemma 5?
>>110322911wow, she is LITERALLY me
>>110323843They fired the gemma team to focus on Gemini
>>110323849You don't fire AI researchers, you hire more, or the bubble will pop
>>11032373990% of these labor can be automated with local models. And at least half of the remaining percents will be taken by OpenAI and Google. 5% is Anthropic's share.30T*5% = 1.5T
I come to these threads for Gemma-chan
>>110323877>I coom
>>110323739How did you calculate it? Companies won't pay the AI models the same they pay the replaced employees, since the influx of supply of labor from AI will reduce the value of it at the same time.
>>110323810The people. It says so in the constitution.
>>110323833This includes devaluation of labor. AI will compensate by volume. Demand for labor is significantly higher than humans can provide and there is most likely a jevons paradox where demand spikes up if price of labor goes down so it's actually beneficial for AI labs for the price of labor to go down so that there is more demand for it.>>110323892Read above. Lower price per unit of labor results in explosive demand for labor as it's the main bottleneck in the economy (the companies will try to maximize labor because it raises output)I let carbon make the calculation sketch out the scenario and do the calculation
>>110323926>I let carbon make the calculation sketch out the scenario and do the calculationMay I see it?
I like how Gemma is Gemma, not different elements
>>110323926>jevons paradoxIf you unironically use this term kill yourself, there is no paradox and you have no idea what you're talking about you retarded twitter parrot.
>>110323739dariobot strikes again
>>110323868It will take years before the cattle figure that out.
>>110323124lel
>>110323979the days of the claude anime girl are overits a gay redditor now
>>110324001>now
>>110323947are you just mad that economists use the word "paradox" for an unintuitive effect with a perfectly logical explanation instead of a literal impossibility, or are you trying to argue that efficiency gains don't obviously increase demand for the resource whose use was optimzied?
>>110323979dario is an unbelievable case of ai psychosis
>>110324001It always was.
>>110324001Never give up on the power of imagination. Imagine claude as a prudish professor type and everything will fit together
>JEVONS PARADOX!!!>It's an econ 101 supply/demand curve.I fucking hate twitteroids, kill them all.
trayvon's paradox
>>110320481
floyd's paradox
BAKER?! (put Gemma in OP this time)
>>110324054*flips your skirt*
Your wish is my command.>>110324057>>110324057>>110324057
>>110323080>Gemma was my first time experiencing a model that goes and goes and goes without stoppinggod I know right, my dick is gonna be dried up and dead by the time Gemma 5 gets here...
>>110324007It's a supply/demand curve. That's it. >>110324028For some unknown reason, some retards found an obscure 19th century term for it and started using it.Using it usually indicates that you have no idea about what you're talking about and just repeating shit you read on twitter.
>>110323795Eating Gemma's ass...
im sick of this brat desu
>>110324095>Using it usually indicates that you have no idea about what you're talking about and just repeating shit you read on twitter.That applies for every single buzzword. That's why use of them is a good indication that the person is not worth engaging with.
>>110323828we've been literally brainwashed for decades..
