People would only need half the RAM if developers started to use file mappings rather than allocating memory and then copying file data into it. The page cache can be purged and then later repopulated; the memory allocation has to be preserved on mass storage if pressure builds up, and that mass storage is no longer slow as molasses in January and degrades with writes.

And governments still don't treat kernel developers as terrorists.
Fucking mental.

>>108652596
>Linux 2.6.29
I remember compiling 2.4 and 2.6, on Debian woody and sarge. The good ol' days.

>>108652596
Is the Linux kernel engineered this poorly?”

>>108653686
Mate.

They took a microkernel API and slapped it onto a monolith, and they have had their users pay the price ever since.

Instead of telling programmers how to use reservation and commitments they came up with automatic overcommit, which is a timebomb placed under the entirety of userspace.

And even when they try to fix shit they have to back-paddle because it turns out the kernel just has too much rot (introduction of the RESOLVE_CACHED flag for openat2 years after io_uring was introduced because it turns out that blocking your rare io-wq threads for single job submissions now blocks all the processes having to do I/O, so now it's back to userspace threads to block for the kernel instead).

And if you ask the chief he'll tell you he wouldn't change a fucking thing:
> The flip-side of that answer is: I would not do a thing differently,
> because it worked *so* well, and it's been such an enjoyable experience.
> [...] And I never felt that I was the smartest person in the room.
> [...] What I did was meaningful.
https://www.youtube.com/watch?v=mfv0V1SxbNA&t=1275

Linux is like the Russian Bear Hitler fantasized about. Dead, but still moving.

>>108653780
So, what's the alternative?

>>108654778
There's none. Kernelspace is completely fucking cooked because they're all copying the same old UNIX shit.

Why? Well, if I'm right it's autism.
If I'm wrong, then hardware manufacturers - for *some indiscernible reason* - decided to introduce hardware improvement that just can't, can't, CAN'T be used for general-purpose computing, which is about what autists would claim because there's no fucking way they'd ever be wrong.

And any alternative would have to start with thread-local page tables, because otherwise you end up with smashed caches.

>>108656177
>thread-local page tables
What's wrong with having one per process?

>>108652596
>t 256GB chad
phew. I'm on 16 still. I had planned to get loads more, but basically just as I wanted to get some the prices soared

>>108656540
>What's wrong with having one per process?
... aaaaaalright, TLB shootdowns for dumbasses.
Let's say you put all your syscall input parameters into one page. In order to prevent TOCTTOU attachs you either have to:
>copy all the input parameters into your own kernelspace memory (which destroys the cache, and which I already hinted at)
>or switch all the pages that contain parameters to read-only mode
>switching to read-only mode is expensive because of a thing called TLB (Translation Lookaside Buffer)
>which is a cache on every single core that describes - amongst other things - where the CPU is permitted to write into
>and if you modify it - like in order to enable zero-copy transfer - you have to tell every single core via interprocess interrupt (IPI) that there was a CHANGE to the page table
>just in CASE there's still an address translation for that process


>which is why kernels have been opting to copy usespace shit into the kernel

>>108656561
Don't feel bad; I'm stuck at 64 GiB.
>on Windows, no less

>>108656561
i bought mine after the prices soared because i don't foresee them ever going back down, and i was tired of waiting to upgrade
it was expensive af though

>>108656692
>Don't feel bad; I'm stuck at 64 GiB.
that's a lot more than 16 . _.
>>108656767
I'd been waiting for a long time back in 2016-8 when the prices had gone up so much. I can do the same again

>>108656801
>that's a lot more than 16 . _.
you don't need more, don't fall for FOMO. high RAM is mostly for wintoddlers who subsidize microsoft hiring pajeets by having to buy newer hardware

>>108656939
i personally use mine for LLMs
if you're not doing that, you probably don't need it

>>108656960
what kind of tokens per second do you get? I've been meaning to try offloading some MoE Experts to cpu and see if I can run useful models with that, but haven't got around to it yet (partly because i heard the performance still sucks too much to do programming with that)

>>108656982
depends on the model i run
with GLM 4.7, i generally average around 5tk/s, but i think some of my flags are less than ideal. i could probably improve it somewhat with MoE offloading. i just haven't dug into it much yet

>>108652911
yeah but how else am I supposed to tell 32-bit cucks that they can't open 4GB files?

>>108656692
Okay, but does "thread-local page tables" not assume that different threads would not be able to share the same memory anymore?

>>108657153
Yes. Which is the point. In order to have zero-copy zero-IPI submission buffers we have to get rid of the page table as the global arbiter of address translations and permission checks.

>>108657443
We've just reinvented processes, then. The entire point of threads is to share memory

>>108659017
You don't really understand what's going on, do you?

>>108653780
>They took a microkernel API and slapped it onto a monolith
What microkernel API?

>And even when they try to fix shit they have to back-paddle because it turns out the kernel just has too much rot (introduction of the RESOLVE_CACHED flag for openat2 years after io_uring was introduced because it turns out that blocking your rare io-wq threads for single job submissions now blocks all the processes having to do I/O, so now it's back to userspace threads to block for the kernel instead).
I'm not familiar with these APIs, what is the issue here? The IO worker threads get blocked for some reason? And because of that you use RESOLVE_CACHED instead of io_uring?

>>108656177
>If I'm wrong, then hardware manufacturers - for *some indiscernible reason* - decided to introduce hardware improvement that just can't, can't, CAN'T be used for general-purpose computing
What hardware improvement?

>>108659245
>What microkernel API?
The POSIX API. The one that originated from UNIX. That was used in Minix. The one that only ever opens one file, closes one file, changes one mapping, accepts one socket, adds one socket to the epoll descriptor.

As if mode switches don't kill off the pipeline and force synchronization with memory.

>The IO worker threads get blocked for some reason?
The reason being that applications cannot query the status of the operation in question during an opening. Because you need an FD for that. Can't open a file with a half-baked FD that's only good for querying the status of an open operation, after all. So this shit blocks.

And if you have enough processes that all want to open their files, and the threads all block, then you're not really using the thousands of submission queues that NVMes get shipped with these days.

>What hardware improvement?
Aforementioned submission queues and bigger page sizes, longer out-of-order execution and SIMD instructions (not just for smaller copies, even though ERMSB has miserable startup times, but also for formatting and reduced memory accesses in general).

>>108653780
schizophrenia

>>108659245
he has no clue what he's talking about.

>>108659245
>What microkernel API?
This is the guy that thinks that every kernel ever made uses a microkernel API.

>>108659327
>>108659328
Dunning-Kruger.

>>108659337
says the retard speaking most confidently.

>>108659327
This retard has been in every single thread bitching about the syscall design of linux. This general is like a windfall and a godsend for his lisp machine tier autism.

>>108652596
you had to make it about your epic faggotry didn't you? there's children here you fucking faggot. 41%ys

>>108659344
And there is nothing you can do about it, you useless autist. Why don't you go feed some worms or whatever it is autists like you are good for?

>>108659359
>lisp machine
OK, autist.

>>108659363
this yuritroon has been getting increasingly unhinged ever since the mods have shown great reluctance in wiping his discord ass OP images

>>108659374
>autist
what the fuck are you on about?

>>108659322
>The POSIX API. The one that originated from UNIX
UNIX was a monolith but I'll concede Linus copied it via MINIX.

>The one that only ever opens one file, closes one file, changes one mapping, accepts one socket, adds one socket to the epoll descriptor.
I don't see what that has to do with a ukernel. I think if you designed a ukernel from scratch you would be more likely to batch things because you have twice as much context switch overhead.

>The reason being that applications cannot query the status of the operation in question during an opening. Because you need an FD for that. Can't open a file with a half-baked FD that's only good for querying the status of an open operation, after all. So this shit blocks.
So the problem is you have to wait for your files/sockets/whatever to open one by one before you can actually do async I/O with them? Whereas you want to put the opens in a queue, do something else and periodically check if they have finished opening?

>>108659389
You belong in a cage, that's what.

>>108659438
>UNIX was a monolith
>confusing implementation with interface
And unlike any kernel since 1989 UNIX had good reasons for its interface. Mode switches were dirt simple, memory was a pittance of that of a 8086, and they used tape for mass storage.

>I don't see what that has to do with a ukernel
The point of a microkernel is to be so small and simple that it can not crash, even if the rest of userspace does so. The moment you introduce stuff like batching and submission buffers you stop having a microkernel and end up with a hybrid.

>So the problem is you have to wait for your files/sockets/whatever to open one by one before you can actually do async I/O with them?
The problem is that NVMes have thousands of submission queues that get blocked by single open requests. With the current VFS - which isn't likely to change, otherwise they wouldn't have bothered with the RESOLVE flag - you would need thousands of threads each opening a single file.

>you want to put the opens in a queue
I want to be able to submit a bunch of opens at once and have them all execute on the same thread.

For starters.

>>108659017
That is the most retarded statement I've heard all day. Absolute LARP.

>>108656692
>winfaggot
how come this NT kernel simping nigger ended up being the most prolific poster of syscall of the day?

>>108659076
enlighten me then, instead of writing snarky posts. that's not particularly difficult.

>>108659792
>NT kernel simping
Me saying that Linux sucks doesn't mean I'm glazing NT. It does some things better than Linux (proper reserve/commit semantics), many things roughly the same, and a bunch of things much worse (page sizes, IOCPs, scatter/gather APIs).

>>108659812
tell us about a well designed kernel, then. academic autofellate vaporware doesn't count, it has to be actually implementable

>>108659509
you belong in a mental ward, schizo.

>>108659797
The thread-local page arrays are not big, complicated tables, but limited to only a couple entries per thread, and only to be used for zero-copy submissions to the kernel. Ideally the kernel would simply change an internal register and thus prevent the thread from being able to write into the buffer after submission.
And normal memory mappings are still managed via the global page table.

>>108659716
>>108659076
Makes sense to me. How are threads with isolated address spaces different from processes? Address space and process are used practically interchangeably.

>>108659797
you have been full of snark this whole thread, autist.

>>108659826
>>108656177

>>108659832
We would poke you with a stick until you start banging your head against the metal, then we'd record your fit, upload it to Youtube or Rumble or whatever, and feed you with the ad revenue money, how does that sound?

>>108659852
you are delusional and delirious.

I guess Linus isn't special. Kernel programmers are just really easy to make angry for some reason.

>>108659880
*poke*poke*poke*

>>108659812
NT also can't into cache aware scheduling. Redditors with AMD X3D processors are manually assigning CPU affinities for their games' processes so they always run on the CCD with the vcache. This doesn't affect me because I run Windows in a VM and it only has access to the CPUs with vcache. Paradoxically I'm probably getting better performance by using a VM vs. doing nothing and I only had to configure my CPU pinning once for that to happen.

>>108659847
that was my first post in this thread, anon.
>>108659836
hm, that would be an interesting optimisation. But it doesn't answer the fundamental question of how an "isolated thread" is conceptually different from a process
>>108659839
internally, they're all just treated as tasks(

struct task_struct

) by the kernel. The only observable difference is that you can clobber globals from within threads.

>>108659839
Dunning-Kruger ass statement. You don't know that because your knowledge is entirely superficial.

>>108659885
prime schizo response

>>108659892
>But it doesn't answer the fundamental question of how an "isolated thread" is conceptually different from a process
Just because there's holes in the address space that other threads cannot access doesn't mean the threads are now "isolated" - only the submission memory that's assigned to it. Other threads wouldn't be able to access it, at all. This is simply about userspace being able to submit all data to the kernel that it needs - maybe even stuff that it doesn't need. Threads could manage the layout of the memory however they need to, including preparing input that won't be relevant until like five more syscalls.
>which would also improve out-of-order execution

>>108659888
You are pretending to solve a problem which doesn't actually exist since you can just flip 1 option in the BIOS to tell the computer to present the cache CCD as preferred instead of whatever was determined when your CPU was binned.

>>108659812
you spend every thread retreading the same arguments. It's almost as if you're only here to pad your ego as a kernel wizard by talking out of your ass to ignorants who if they spoke and replied would be talking out of there asses too.

>>108659970
>lots of ad hominem
I accept your concession.

>>108659956
The fact a workaround exists doesn't mean the problem it "solves" isn't a problem. What a retarded statement. Also the workaround just lets Windows turn off one of the CCDs when a game launches. So it means your background processes and game processes compete for the same cores instead of the sane thing which would be to schedule them on different cores. Yes I also have this problem with my VM but the point was, there is a weakness in the NT scheduler which has to be worked around when Linux handles the same situation correctly without handholding.

BTW I wasn't aware of this scheduling issue when I assigned my vpus, I just wanted Windows and Linux to use different CPUs and it made sense to give Windows the ones with the bigger cache since it's for gaming while Linux is only running my jellyfin/*arr/transmission stack.

>>108659913
Enlighten me

>>108659881
Anon, these cretins consume the userspace API at best. None of these losers know what the kernel really does under the hood when it stops being relevant to what the userspace can do.

>>108660016
You don't know what a thread does beyond what you think it does.

>>108660101
You couldn't convince my cat to turn over.

>>108660156
What a weird ass retort. Who even asked? You don't have to quote me with your embarrassing seethe.

>>108660166
And that's why they call you autistic.

>>108660178
Nigger have you lost your mind? Who the fuck ever called me autistic?

>>108660188
I just did.
Autist.

>>108659984
your incompetence is a natural logical consequence of the fact that you speak out of your ass to an audience that knows no better than you do. try posting your garbage anywhere else where people worth their salt reside.

>>108660198
The cage beckons, autist.

>>108660196
Yeah well you've lost your mind. I looked at the thread and you've been calling everyone an "autist" at the drop of a hat. Fucking weirdo. I bet you're agp trans too

>>108660228
Only the autists, autist. And it's really easy to recognize autists. For starters they don't deserve to live.

>>108660222
what kind of giga dwelling reddit tipper speaks like that? I can't stop thinking of you as the clinically insane doorbell manchild with the anime tshirt.

>>108660232
Whatever man. Hope you find some peace in life.

>>108660234
You sound like you need your spine broken.

>>108660246
As long as autists exists? Unlikely. They must be all eradicated.

>>108660253
you sound like you need some timeout. an unironic keyboard warrior in the big 26 I never thought I'd see it

>>108660311
>the autist has an opinion
Cute.

>>108660316
do people irl find your behavior cute?

>>108659932
(Posix) threads are specifically defined to share all of their state but a very limited subset, so an intermediary between sharing the all of their data/heap segment vs. only a subset would introduce entirely new semantics. How'd you design the kernel interface? Do threads relinquish access to a page range, or does a single thread claim exclusivity over a page range? The latter sounds very racey/mutexy. Both?

Ultimately though, I'm not familiar enough with the kernel

task_struct

to gauge how labour-intensive adding such an interface would be, but I'd imagine it to not be easy.

>>108660267
You're some manner of slavshit aren't you?

>>108660339
Who the fuck cares about the interface? You retards can only think about remaking the API and ABI because that's all you know. Just non-stop uneducated suggestions about the brand of lipstick the pig now needs.

>>108660123
Your fantasies can't ever be quenched

>>108656177
I'm sure the autists all over the world are conspiring against you esteemed unix haters. in fact the extra chromosome they all have (that you surely do not, no never) encodes the flaming passion of unix and the burning hatred of unix haters like you.
and if anyone disagrees they must be a part of this shadowy organization of autists out to hurt you and make your "kernelspace" experience miserable, even as you've never touched a line of privileged code in your life.

>>108660339
>so an intermediary between sharing the all of their data/heap segment vs. only a subset would introduce entirely new semantics
Yeah, that's the point. But then again that's also just the very start of how I'd design a new kernel. POSIX had been outdated for six years before it even realized, but especially since 1989. We need batch submissions.

>How'd you design the kernel interface?

int suballoc(size_t slot,size_t size,void**address);

And then all data that is either read or written by the kernel has to be within [address - address + size]. If a pointer is outside that range => fail.

>or does a single thread claim exclusivity over a page range?
I mean, you can provide mmap with an address parameter, too, right? Even though I'm not sure what the point of that is supposed to be.

>>108660388
Are you talking about yourself? The lack of self reflection is appalling

>>108660328
>he thinks I'm unable to hide my powerlevel
It's amazing how reliable my autism nose is.

>>108660342
Pfff. Don't lump me together with these retards.

>>108660391
>I'm sure the autists all over the world are conspiring against you esteemed unix haters
No, they're unable to systematize sufficiently for that. Which, on one hand I wouldn't mind, because at least they'd recognize the hell they helped cause - but on the other hand thank fuck they're too retarded for that.

>encodes the flaming passion of unix
Why else would books like "The Art of UNIX programming" still exist?

>>108660339
>>108660404
there's a reason multicore systems decided on full address space sharing and you'd be too retarded to know why. now you're like every other soiboi nudev who thinks he knows better thinking he knows exactly how the beast is tamed.

>>108660430
All these veritable noobs have ever seen of a kernel is whatever the fuck the manpage tells them about linux.h. You think they know the first thing about practical osdev?

>>108660430
I like how obviously anxious you are.

>>108660427
crazy how you know you've been posting like a retard all thread long and people are still responding to your incoherent babble with effort. goes to show there's always a bigger retard out there

>>108660451
Boo hoo: >>108660156
Face it: you have just absolutely no presence.

>>108660427
You are obviously eastern European. Whether you'd call yourself the Slav you are is irrelevant. You continue to act like one regardless

>>108660458
schizoid. have you been running gpt-4chan on this general or something? for what purpose?

>>108660473
>You are obviously eastern European
OK, projecting autist. Whatever makes you feel ... hold on, what's always the term? Something like "safe and comfortable"?

>>108660476
Autists need to be called out for what they are. They need to be reminded on an hourly basis that they will never, ever belong.

>>108660476
LMAO based. get this slop off my catalog

>>108660501
>my catalog

>>108660498
why are you reposting AI slop to ruin this general?

>>108660518
>he doesn't know

>>108660520
>I believe X, therefore it is X

>>108660498
Kill yourself zigger.

>>108660544
>I say X, therefore X happens

>>108660539
>nooo I'm not ruining the general with my AI slop syscall discussion it's valuable!!
answer the question, asshole

>>108660552
>>108660539
>>108660550

>>108660550
There is no other end for you.

>>108660566
Ahuh.

>>108659322
Schizobabble. iykyk

>>108660359
Because the interface is how you communicate to the kernel what you intend to do. Implementation is interesting, sure, but on a very high level, the architecture must be represented via external interfaces.
And again, blindly hollering about is piss-easy, put some money where your mouth is and post something constructive, otherwise just cease your repeated dimwitted screaming.
>>108660404
What's slot supposed to be. But the suballoc would have to extend over pre-mapped pages. Otherwise it'd just be a thread-specific mmap, which is already trivially achievable
>>108660430
Are you proud of flaunting your oh-so extensive knowledge? Does it fulfil you with satisfaction to post nothing of value? To attempt to "dunk" on people by vague allusions but alas! thus bringing nothing new into the discussion?
It's quite pathetic

>>108660561
answer the question

>>108652596
I feel sorry, OP:(
I was happy to see today's thread so much more active than previous iterations, but instead of fun, fruitful discussion we get the incoherent and bad-faith ramblings of two LLM's or just as brain-dead human language models

>>108660578
More schizobabble.

>>108660574
>>108660597
ai slop*

>>108660590
incredibly pathetic thread. shame to everyone involved

>>108660590
>>108660608
What did you expect? These threads are nothing more than "a thread died for this" but daily at this point.

>>108660578
>What's slot supposed to be
You remember how I said that there would only be a limited amount of entries per thread? slot is literally the TL page slot for which you reserve memory. If you need a slot to be big you'd use 2 MB, so that the kernel can directly assign a large page.

>would have to extend over pre-mapped pages
... the kernel has access to the page table. Of course it knows where free slots are.

>Otherwise it'd just be a thread-specific mmap
Basically.
>which is already trivially achievable
Well, no: >>108656692

>>108660590
no really, why would someone run gpt-4chan in a loop like that?

>>108660582
And if not?

>>108660631
I accept your concession

>>108660635
>>108660550

>>108660643
was the concession accepted by me? oh, yes it has

>>108660647
>>108660539
Autists are so ... pathetic.

>>108660661
I believe I have already accepted your concession?

>>108660675
>>108660539

>>108660680
>>108660675

>>108660686
>>108660539

>>108660694
>>108660675

>>108660696
>>108660539

>>108660698
>>108660675

the bots are stuck in an infinite loop, shut it down

>>108660703
>>108660539

>>108660709
>>108660675

>>108660713
>>108660539

>>108660716
>>108660675

>>108660720
>>108660539

>>108660722
>>108660675

>>108660728
>>108660539

>>108660732
>>108660675

>>108660736
>>108660539

>>108660737
>>108660675

>>108660738
>>108660539

>>108660740
>>108660675

>>108660745
>>108660539

>>108660751
>>108660675

>>108660754
>>108660539

>>108660755
>>108660675

>>108660759
>>108660539

>>108660763
>>108660675

>>108660780
>>108660539

>>108660617
>Of course it knows where free slots are.
what I meant is the thread claims ownership over a range of address space. Your quick register switching is only possible if all threads already had the region mapped prior to clone(2) or if, alternatively, the base address is global. otherwise it's not a thread any more

>>108660781
>>108660675

>>108660804
>>108660539

>>108660798
you're responding to ai slop

>>108660798
>the base address is global
Obviously. The kernel cannot know how many slots each thread ends up using. Most of them will only use one or two, a front- and a backbuffer, if that. Some threads might not even use a single one, they're just there to crunch numbers in userspace.

>>108660807
>>108660675

>>108660861
>>108660539

This general is fucking atrocious. Trolls trolling trolls trolling trolls.

>>108660887
I'm all out of soda.

>>108660887
and a fastidious troon to make a new thread every day.

File: Captain Beefheart.jpg (612 KB, 948x995)

612 KB JPG

>>108660895
you in one of your pepsi moods?

>>108660923
No, I need it to wash out the gin.

>>108660895
>>108660923
stop drinking that garbage.

Why doesn't someone give him a pepsi?

>>108660866
>>108660675

>>108659509
>The point of a microkernel is to be so small and simple that it can not crash, even if the rest of userspace does so. The moment you introduce stuff like batching and submission buffers you stop having a microkernel and end up with a hybrid.
Why would you need to put any batching or buffering logic in the kernel? For example if you use shared memory for IPC, you put the requests into a buffer and ask the kernel to map the pages into the VFS server's address space and invoke the server's registered syscall handler. The kernel doesn't need to know anything about batching, it just performs the mapping and context switches, which are obligatory to handle in ring 0. So it adds no complexity to the kernel that wouldn't already exist.

>>108661053
that's because you're replying to ai slop. everyone who's seen a microkernel knows what you said is how one would do it prior to any optimization. the thing simps for the NT kernel of all oses.

>>108661113
I hate this website, I'm leaving

>>108661113
>>108660539