>>109568780
i think the compression of any file is limited by its entropy, which is a measure formalized in information theory, and since this is a property of the file itself there's a hard limit on how good your algorithm can be. but i could be wrong since i didn't research it as much as i would have liked. if anyone knows i'd be fine with being lectured desu

https://en.wikipedia.org/wiki/Kolmogorov_complexity#Compression

>Is it just mathematically impossible for compression to get better

Past a certain point, yes. The magic lays in determining that point.

>>109568780
Just check the quality of the programmers over time.

What we need to do is start recording video so that the bits are all very early sequences found within pi. Then it will all compress really well.

>>109568843
Then drop math and use something else. Simple as.

NOTHING is impossible.

for many kinds of compression we kinda just did most of the obvious stuff really quickly and most practical data compression uses require a balance between speed and ratio, so spending disproportionately more effort on tiny ratio bumps just doesn't make sense.
it's why modern algorithms like zstandard got popular quickly despite not even compressing as much as older formats, what makes zstandard so popular is because it has /good-enough/ ratios, meaning about as well as gzip, while being /way/ faster than gzip.

it's like do you use zpaq and make the file 10% smaller but take 1000x more time to do it? probably not.

there is a theoretical maximum any file can be compressed to as well, so it's not a case of "not knowing what you don't know", either. actually reaching that maximum with any reasonable amount of time and complexity for any file is difficult. getting close enough faster is what most algorithms aim for since they have the most practical use

>>109569114
should add i'm just talking about general purpose lossless compression here (like zip/7z/xz/zstd/gz/etc).
domain-specific lossless compression (like say flac) is a bit different, and lossy compression (h.264/av1/vorbis/mp3/etc) opens up many, many more options since those are as much subjective as they are mathematics.

>>109568780
ANS powering your zstd compression format was invented in 2014, 12 years ago.

>>109569114
this and files have gotten bigger, what was tolerable compressing a 1GB file may not be tolerable compressing a 100GB file even with cpus getting faster

>>109568997
Unless you find a better way to store the files than 1s and 0s you will not be able to fit more actual data into the same space, especially if the video isnt repetitive and its background changes a lot.

File: 1768029601582856.jpg (51 KB, 750x766)

51 KB JPG

>>109568780
Because of low hanging fruits. You can't just keep compressing shit harder. It's like asking why we haven't found any bigger animals than the blue whale.
>inb4 seething

LLM technology is literally compression. We just haven't used it widely as a compression algorithm yet.

>>109570329
dictionaries in compression has been a thing for ages. zstd supports dictionaries if you have data which benefits from it

You don't need better compression to maximizing your local storage when all types of entertainment gets streamed into your computer

LLM is a lossy compression.

>>109568780
>advancements in compression in like 30 years?
Whatchew talkin' 'bout willis?
They literally have competitions for this every year.

>And lossless compression hasn't gotten better since the 90s
You can naturally be backing up this statement with some credible facts?

>Is it just mathematically impossible for compression to get better
I wouldn't say impossible. But the problem with improvement is - if it is indeeed legimate improvement - that it nudges the bar to clear that little bit higher.

When the bar is set real low, nudging it relatively large distances is a lot easier. When it's 99% to the top it gets it real hard to move it 10%. But hoping for 1% isn't unreasonable.

>>109568816
>but i could be wrong
No, there's definitely a hard limit.
Where that limit is has a little scope for argument however.

>>109568997
>Then drop math
And replace it with, what, precisely?

Do you know *why* the math is there?

>NOTHING is impossible.
except you taking the time to think, it would appear.

>>109569126
>general purpose lossless compression
>domain-specific lossless compression (like say flac) is a bit different
Same rewls apply.
Narrowed scope of application can create opportunity to optimise for said application - but it's still the same rewls.

>those are as much subjective as they are mathematics.
No. It's only math.
The computer doesn't work by vague feels. It's 0 or 1. Mebbe a 2 if you're quantum.

>>109569837
>even with cpus getting faster
Typically, they have 'accelerator' circuits - essentially ASICs to handle that specific math. Or more accurately, the precursor math. One would imagine 'more cores' here to throw more loads in, in parellel, would be the 'next step' if common file sizes become too uncumbersome.

>>109571551
And worst, you can't (reiably) quantify the loss.

>>109572048
>You can naturally be backing up this statement with some credible facts?
Well is there anything more efficient than FLAC for lossless audio?

>>109568780
>New video codecs just find ways to make compression artifacts less perceptible
This is basically the same thing you're asking for. Fewer artifacts means that you can compress more with the same level of perceived quality.

>>109572048
>Narrowed scope of application can create opportunity to optimise for said application - but it's still the same rewls.
yea, i added that in last second because i realised not all lossless compression is general-purpose, but probably shouldn't have brought it up without the intention of going into any detail about it.

>>109572096
not him but actually there might be. i don't keep up to date with lossless audio compression but i do know monkey's audio and optimfrog are quite close. last i checked optimfrog is closed source and monkey's audio is notably slower than flac, so flac comes out as the best balance. some samples may be better compressed in another format, but it's hard to beat flac's compatability, especially nowadays. even if your .ape album is a bit smaller, how many players support that compared to .flac do you think?

>>109572096
>Well is there anything more efficient than FLAC for lossless audio?
Tried .zip?
By which I means actually as the audio codec. You rip the CD like that and it'll eat less space than .mp3... Naturally that'll self-limit what can play it, but is it my fault the amplifier manufacturer coudn't concieve such a thing?

Look at me, showing my age talking of optical media and other cutting edge tech of the '90s.
/me hangs head in shame.

>>109572212
i'm drunk myself but all i want to ask is if you're drunk

the audio version of .zip? what? that'll use less space than mp3? how? limit what can play it? what music players accept .zip?
cd? cutting edge in the '90s? it came out in '82, there was nothing cutting edge about it.

>>109568780
The most recent advancement I know of is 'superresolution'

i.e. youve seen something so many times, with just a shred of it you can guess the rest

Other than that Idt theres much room for growth

>>109568780
There's a theoretical limit to how much anything can be compressed, and they're already pretty close to it.

>>109568780
Because the big guys need to sell hardware

>>109572575
>if you're drunk
No.
Would it help? I've got plenty o 99.98% Isopropyl kickin' about. I mean sure, it's technically an industrial solvent, but it's cleaner than what you can get from the shelf - and diluted to at least 50% in my kawphee it'll be somewhere near a shot o whiskey erry sip...

>the audio version of .zip? what?
No. The .zip compression algo used as an audo codec.

You see. The problem with not understanding how the toaster works is it curses you to a life of only ever envisioning the toaster to heat bread.

>that'll use less space than mp3? how?
Better compression?
I thought that was the question?

>limit what can play it?
Kinda. How many off the shelf fings you seen seen listing ".zip" as a format they'll play OOTB?
Tho in fairness plenty say '.mp3' and don't actually support all of the sub-flavours...

>what music players accept .zip?
They don't unless you make them, kinda the point of the initial statement.

>cutting edge in the '90s?
>there was nothing cutting edge about it.
It was when you was rippin' 'em to .mp3... Or in a select few cases, .zip.

Do you actually remember a time before .mp3?
I swear. The yout o t'day really don't know how good they gottit...

>>109572706
The disposable mentality is literally by definition unsustainable. It's rapidly approaching: Evolve or die.
They wil ofc yield to greed and wring everything out of the simpletons they can, but anyone that actually intends on survivng the transition has already started to adjust. Look how many 'names' dumped into the open source car project. When you build a car with a lifecycle that spans generations you make far more money on parts than you will actual cars, and if everyone's parts just snap in like lego then everyone can keep the existing circle jerk that makes them feel special for a little longer...

>>109572878
>Would it help? I've got plenty o 99.98% Isopropyl kickin' about. I mean sure, it's technically an industrial solvent, but it's cleaner than what you can get from the shelf - and diluted to at least 50% in my kawphee it'll be somewhere near a shot o whiskey erry sip...
just in case you aren't joking, please don't drink isopropyl alcohol, it's not the same as ethanol alcohol which is the kind of alcohol that is drinkable.
>No. The .zip compression algo used as an audo codec.
>You see. The problem with not understanding how the toaster works is it curses you to a life of only ever envisioning the toaster to heat bread.
you can compress raw audio with the DEFLATE algorithm. it'll do alright but probably not as well as flac
>Better compression?
>I thought that was the question?
don't expect better compression (ratios) with a lossless format over mp3
>Kinda. How many off the shelf fings you seen seen listing ".zip" as a format they'll play OOTB?
approximately none to be honest (desu)
>Tho in fairness plenty say '.mp3' and don't actually support all of the sub-flavours...
i don't think there are any official "sub-flavours", though i've heard of things like mp3-pro and an integer-only form of mp3 but i don't expect hardware players to support anything but plain ol' MPEG Layer 3
>Do you actually remember a time before .mp3?
sure, i owned cassette tapes

>>109572878
oh also even if it was drinkable, i distill my own alcohol and near-pure alcohol curdles the milk in coffee instantly.

ask me how i know

>>109573045
>just in case you aren't joking
I'm not. But I did get confuddled. Tub deffo says: ethanol.

>don't expect better compression (ratios) with a lossless format over mp3
Then how can I .zip an album as .mp3 and end up with a smaller file than it's consituent parts?

>official "sub-flavours", though
So you don't get a drop-down list of bitrates when you're recording audio?

>sure, i owned cassette tapes
Props.
But have you seen someone with an MP3 player plugged into their mixer, and using it to sample to c90? /me shudders

>>109573063
>curdles the milk in coffee instantly.
Good thing I drink my kawphee so black it steals my bike.

>i distill my own alcohol
Flatmate had a spell with a still. It's all well and good, but for what you can buy it outta lab, it's not worth the sugar...

>>109568780
Ironically enough BECAUSE of computer science.
It sucked up all the mathematically inclined people so maths is undergoing unsustainable Brain drain.

>>109572048
>No. It's only math.
Feel free to specify a quantization table best-suited for human perception from first principles, then. Any reference to the natural world immediately disqualifies your answer.

>>109573591
>best-suited for human perception from first principles
Isn't the whole point of the compression entirely because it's already unsuitable for human perception?

>Any reference to the natural world immediately disqualifies your answer.
And your evidence that math - the laws of physics specifically - fail to govern the natural world in it's entirety?

>>109568780
Posts that could be answered with a google search should be deleted
https://en.wikipedia.org/wiki/Comparison_of_audio_coding_formats

>>109573836
couldn't help but look into the fabrice bellard (he's just the guy who made tcc, qemu, and ffmpeg) one.
of course it's a transformer based codec, that was inevitable

File: wpeg.png (71 KB, 1920x591)

71 KB PNG

>>109568780

We knew about the Shannon limit back when computers were still using vacuum tubes.

File: 2 minutes and 13 seconds (...).png (80 KB, 102x102)

80 KB PNG

>>109574002
fucking hell that works better than expected... slow as balls but it did say it would be

> magick 2\ minutes\ and\ 13\ seconds\ of\ audio\ btw.png rgba:decode.tsac
> truncate -s 81833 decode.tsac
> ./tsac d decode.tsac decode.wav

>>109574193
decoded file: https://litter.catbox.moe/itkfbdfdznh6qu8r.wav
79.9KiB file btw. insane.

>>109568780
I'll be honest

I stopped caring about compression and encoding the day I bought my first 10TB HDD

I don't care about AV1 AV2 any of that shit

compression is for Youtube

>>109574342
a true hoarder will expand to fill the available space. my roommate has a nas with 6x10TB hdd's in it in raid6 (~40TB logical space) and it's been full for years, only clearing enough space during a cleanup to make room for more stuff