>>108571933
Zstandard, or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library.

>>108571952
thank you Grok. fuck off and kys

Go and read wikipedia or something

>>108571933
it's good desu, I use it for zswap

Bot thread

>>108572773
>>108572773
Developed by IDF Israelis. I don't think so pal

>>108571933
The algorithm isn't backdoored, individual implementations may include backdoors as with all things.
Just be mindful about application and data separation and use virtual machines, containers, and sandboxing utilities like firejail and bubblewrap to control what programs have access to what data and resources on your system appropriately.
If you're on windows or mac os then ZSTD being backdoored is the least of your concern because your OS is already backdoored.

File: 1754533644211086.gif (370 KB, 200x200)

370 KB GIF

>>108573276
Okay Shlomo

>>108571933
read the fucking source code. you can look and see if it's backdoored AND discover how it works at the same time. why are you so helpless? or would you rather enjoy your stupidity and just post conspiracy shit all day?

File: compression.png (13 KB, 612x333)

13 KB PNG

>>108571933
Here is the rundown:
>Zstandard is made by Yann Collet, who is known in Open Source for his earlier work on LZ4.
>Facebook hired Yann Collet based on that work, and sponsors/hosts Zstandard development.
>Archtrannies switched to it because it is "zomg bleeding edge!" https://archlinux.org/news/now-using-zstandard-instead-of-xz-for-package-compression/
>anti-systemd chuds hate it: https://sysdfree.wordpress.com/2020/01/04/293/
Arch's testing is incorrect, LZMA usually gets around 5% smaller sizes than Zstandard when you don't use tranny options like --ultra, and --ultra is still 5% worse than xz -9 which you also shouldn't use.
Package sizes are more important because it is usually network that is the bottleneck, especially on a rolling-release binary distro.

>muh xz backdoor
Facebook could easily backdoor libzstd.so, which links to everything just like liblzma.so.
But you shouldn't use xz/lzmautils for other reasons, use lzip instead, made by GNU Developer Antonio Diaz Diaz.
https://www.nongnu.org/lzip/xz_inadequate.html

>kernel compression
See pic related.
Zstandard would be worse than those in terms of DT+LT time, but achieve a lower vmlinuz size.
LZMA achieves even smaller vmlinuz but at the cost of rather slow boot times.
It's worse than LZO/LZ4 but better than gzip roughly.

>>108574215
>Arch's testing is incorrect
dumb retard
>Recompressing all packages to zstd with our options yields a total ~0.8% increase in package size on all of our packages combined, but the decompression time for all packages saw a ~1300% speedup.

>>108574235
Yes it's incorrect, try it for yourself.
This is my results with zstd -19 versus xz -6 (the default).

 79M Apr 10 11:39 firefox-149.0.2.tar.xz # mozilla orig
298M Apr 10 11:40 firefox-149.0.2.tar
 87M Apr 10 11:40 firefox-149.0.2.tar.zst # zstd -19
 82M Apr 10 11:40 firefox-149.0.2.tar.xz # xz -6 recompression

Archtrannies were using --ultra tranny settings, the "fair" comparison would be --ultra settings versus xz -9 which would yield a similar spread of around 5% or higher.
But don't use those retard: https://github.com/facebook/zstd/issues/435

>>108574235
>>108574360
For some laughs, see how retarded Archtrannies truly are:
https://www.phoronix.com/news/Arch-Linux-Bizarre-Zstd

>>108574360
>I get different results with different settings
wow, you're really smart

>>108574382
>Zstd appears to have more than one build system supported by upstream. Relevant to Linux are:
>* Plain Makefile (what Ubuntu uses to build)
>* CMake (what Arch uses to build)
>* Meson (not sure who uses this but I tested it for completeness)
try reading, it's an upstream issue

>>108574215
>Package sizes are more important because it is usually network that is the bottleneck, especially on a rolling-release binary distro.
I would think 1300% longer to decompress would be worse than a few seconds extra download time. Also more CPU intensive I'm assuming.

>>108571933
>backdoored
It's a fucking compression algorithm that's in most other applications
You don't know how computers work
Fuck off

>>108574385
zstd -19 is the highest you can go without using tranny --ultra options which increase ram usage during decompression
That's the equivalent to xz -6. xz levels 7, 8, and 9 use more ram on decompression.
Archtrannies use the default of 3 in makepkg.conf LOL
While the tranny testing of "zomg only 0.8%" obviously compared --ultra to baseline xz.

>>108574397
1. Building with CMake is retarded
2. Archtrannies don't test their builds so they didn't even notice their shit running slow, they had to have MICHAEL LARABEL tell them

>>108571933
Corporations process a shitload of data because they capture each little move of your mouse that happened while you had the photo of your classmate with big tits on screen for two minutes. They really benefit from transparent way to shrink easily compressible data for storage and network transfer (if you buy 10 very expensive servers with very expensive network cards instead of 50, it's quite a save for the company), but don't want to spend extra time on already compressed items. They also don't want to spend human time on analysing all kinds of data manually to tweak the algorithm. That's why they develop one-size-fits-all solutions that can be both as fast as possible when minimal lag for compression and decompression is desired, and also as good as possible when CPU time is in abundance (long term cold storage), all through the exact same pipeline. zstd was not even the first such project.

https://fastcompression.blogspot.com/2015/01/zstd-stronger-compression-algorithm.html
https://engineering.fb.com/2018/12/19/core-infra/zstandard/
https://gregoryszorc.com/blog/2017/03/07/better-compression-with-zstandard/
The last one has a graph that explains how it covers everything, or at least tries to be not worse that other methods.

>>108574215
>Package sizes are more important because it is usually network that is the bottleneck, especially on a rolling-release binary distro.
nobody cares about your thirdie dialup connection

File: 1751476392574884.jpg (54 KB, 1080x1080)

54 KB JPG

>>108573873

>>108574505
thank you very much. your contributions have been truly insightful and helpful

>>108571933
look at the code
if you can't, then always assume yes

>>108571956
>Grok

>>108574408
>>108574360

# time xz -kd -T0 -f firefox-149.0.2.tar.xz
real    0m0.795s
user    0m2.827s
sys     0m0.100s
# time zstd -kd -T0 -f firefox-149.0.2.tar.zst
real    0m0.486s
user    0m0.449s
sys     0m0.108s

So for a 79-82M file, you save 2.4 seconds.
We can now compute a "breakeven" connection speed. Do the math that's around 20mbps.
Lower than that and LZMA wins on pure speed, higher than that Zstandard wins on pure speed.
Even if you have a good network connection, there are other considerations that work in favor of LZMA:
>if network reliability is an issue (mobile data or wifi), that gives an edge to LZMA
>if a mirror server is under heavy load, it may throttle your speed, giving an edge to LZMA
>if a server is serving smaller file sizes, that reduces the overall burden on the server, favoring LZMA
>bandwidth is usually a scarcer resource than cpu, so cost-adjusted shifts towards LZMA
>account for storage costs, especially for example, RAID on server-side, shifts benefit towards LZMA
>if you have unlimited bandwidth and storage, then actually both LZ4 and LZO will overtake Zstandard in pure speed for this use case
Basically, Zstandard is not really competing with LZMA or LZ4 which are both best in class for their use case, it's in the middle ground, where it's competition is mostly gzip/zlib which it edges out, but without the level of adoption and availability that gzip has.
One can argue that 20mbps is a "low enough" breakeven to where zstd -19 is a reasonable compromise between the extreme use cases, but I would argue that network reliability and server load probably outweigh that.

File: img-2026-04-11-06-15-05.jpg (668 KB, 2880x2816)

668 KB JPG

>>108577356
dumb retard

>>108571933
>backdoored compression algorithm
unless you are talking about the "XZ backdoor" you are actually fucking braindead.

File: team-emmanuele-bassi-396651958.jpg (21 KB, 460x460)

21 KB JPG

>>108579673
Your image is exactly proving what I am saying.
>LZMA, Zstandard, LZO/LZ4 are Pareto frontier
>Strongest use case argument for the extreme points of the Pareto frontier, not the middle part
Graph does not represent Utility when you consider use case. Convexity is not guaranteed, e.g. take a log.

>>108573873
grok why am I so helpless?

>>108580693
>Strongest use case argument for the extreme points of the Pareto frontier
wrong, dumb retard

>>108571933
Don't care. Still using gzip.

>>108580771
Explain how I'm wrong.
Use case in this domain is such that only one of your variables is useful.
1. For networks, I care more about file size due to bandwidth constraints.
2. For kernel/initramfs, I care more about decompression time for speedmaxxing.
I don't really see the benefit for compromising file size in the first use case in exchange for decompression speed.
I don't really see the benefit for compromising decompression speed in exchange for file size in the second use case where I want my bootup to be as fast as possible.
The use cases erase the middle part by stratifying the utility function.
You need to construct a cost function for each use case to prove me wrong.

File: dumb retard.jpg (540 KB, 2880x2744)

540 KB JPG

>>108580807
>1. For networks, I care more about file size due to bandwidth constraints.
that sounds very NA coded, here in the first world we have gigabit speeds
>2. For kernel/initramfs, I care more about decompression time for speedmaxxing.
see pic

>>108580825
>that sounds very NA coded, here in the first world we have gigabit speeds
Again, I laid out the points here >>108577356
>if network reliability is an issue (mobile data or wifi), that gives an edge to LZMA
>if a mirror server is under heavy load, it may throttle your speed, giving an edge to LZMA
>if a server is serving smaller file sizes, that reduces the overall burden on the server, favoring LZMA
>bandwidth is usually a scarcer resource than cpu, so cost-adjusted shifts towards LZMA
>account for storage costs, especially for example, RAID on server-side, shifts benefit towards LZMA
>if you have unlimited bandwidth and storage, then actually both LZ4 and LZO will overtake Zstandard in pure speed for this use case

>see pic
See the pic here: >>108574215
See: https://events.static.linuxfound.org/sites/events/files/lcjpcojp13_klee.pdf
That's apprximately a 1:1 tradeoff between file size (LT) and decompression time (DT).
Now we can see why the chart is extremely misleading:
>DOES NOT start the x-axis at ZERO.
>The inclusion of LZMA (which is obviously bad for this use case) makes it hard to compare the Y axis between LZ4 and Zstandard
Nevertheless, we will try to use it.
zstd -8 and zstd -7 appear to offer less than half the file sizes of LZ4, while offering more than double the decompression times, making LZ4 win out.
I would need the raw numbers to compare zstd -1 versus LZ4.

File: lzbench.png (40 KB, 829x851)

40 KB PNG

>>108580825
>>108580897
Was playing around with lzbench.
Tried to choose a representative dataset and machine.
Match the desired use case (reading kernel from disk and decompress).
500 MB/s in pic related. Representative of an HDD I have on my machine.
As you can see LZ4 wins over Zstandard.
My SSD is closer to 2000 MB/s read speeds so actually no compression probably wins on SSD.
https://morotti.github.io/lzbench-web/?dataset=silesia/mozilla&machine=desktop#results

>>108581261
>zstd version 1.3.4
>released this Mar 27, 2018
dumb retard

>>108574418
To fair to schizo retards, it could have theoretically been designed to be insecure to implement. It isn't but theoretically.

>>108581312
>my algorithm got magically faster because it just did okay!!!!

>>108581424
>https://github.com/facebook/zstd/releases/tag/v1.4.0
>Zstd's fastest compression level just got faster! Thanks to ideas from Intel's igzip and @gbtucker, we've made level 1, zstd's fastest strategy, 6-8% faster in most scenarios.

>https://github.com/facebook/zstd/releases/tag/v1.4.1
>This release also includes some performance improvements, among which the primary improvement is that Zstd decompression is ~7% faster,

>https://github.com/facebook/zstd/releases/tag/v1.4.4
>Decompression speed has been substantially improved, thanks to @terrelln. Exact mileage obviously varies depending on files and scenarios, but the general expectation is a bump of about +10%.

>https://github.com/facebook/zstd/releases/tag/v1.4.5
>Decompression speed has been improved again, thanks to great contributions from @terrelln.
>For x64 cpus, expect a speed bump of at least +5%, and up to +10% in favorable cases.
>ARM cpus receive more benefit, with speed improvements ranging from +15% vicinity, and up to +50% for certain SoCs and scenarios (ARM‘s situation is more complex due to larger differences in SoC designs).

>https://github.com/facebook/zstd/releases/tag/v1.4.7
>This release includes optimizations that significantly speed up decompression of small blocks and small data. The decompression speed gains will vary based on the block size according to the table below:

>>108581424
>>108581623
>https://github.com/facebook/zstd/releases/tag/v1.4.9
>2x Faster Long Distance Mode

>https://github.com/facebook/zstd/releases/tag/v1.5.0
>1.5.0 introduces a new default match finder for the compression strategies greedy, lazy, and lazy2, (which map to levels 5-12 for inputs larger than 256K). The optimization brings a massive improvement in compression speed with slight perturbations in compression ratio (< 0.5%) and equal or decreased memory usage.
>The decompression speed of data compressed with large window settings (such as --long or --ultra) has been significantly improved in this version. The gains vary depending on compiler brand and version, with clang generally benefiting the most.

>https://github.com/facebook/zstd/releases/tag/v1.5.1
>PRs #2749, #2774, and #2921 refactor single-segment compression for ZSTD_fast and ZSTD_dfast, which back compression levels 1 through 4 (as well as the negative compression levels). Speedups in the ~3-5% range are observed. In addition, the compression ratio of ZSTD_dfast (levels 3 and 4) is slightly improved.
>Our Huffman code was significantly revamped in this release. Both encoding and decoding speed were improved.
>Overall impact on (de)compression speed depends on the compressibility of the data. Compression speed improves from 1-4%, and decompression speed improves from 5-15%.

>https://github.com/facebook/zstd/releases/tag/v1.5.4
>This release has accumulated a number of scenario-specific improvements, that cumulatively benefit a good portion of installed base in one way or another.

>https://github.com/facebook/zstd/releases/tag/v1.5.5
>Speed improvements of middle-level compression for specific scenarios

>https://github.com/facebook/zstd/releases/tag/v1.5.7
>Enhanced Compression Speed for Small Data Blocks

>>108581623
>>108581632
https://github.com/inikep/lzbench has zstd version 1.5.6
CONFIG_KERNEL_ZSTD uses zstd --ultra 22 so that it can decompress in one pass without allocating buffer.
Throughput: 1073 MB/s, Compression Ratio: 24.69
CONFIG_KERNEL_LZ4 uses lz4hc -9. Throughput: 3527 MB/s, Compression Ratio: 36.75.

From this you can compute a different breakeven this time, a breakeven disk read speed at which LZ4 overtakes Zstandard, that speed is around 170MB/s.
That means if you have a SSD, you should use LZ4 for Kernel compression.
Zstandard might edge it out on a slow ass HDD.

>>108580825
>that sounds very NA coded, here in the first world we have gigabit speeds
By the way, as per your logic, 170MB/s is 1.3 gigabit, so on your le epic europoor gigabit internet you should consider LZ4.
Do you now see how retarded you sound? And why LZMA makes the most sense over networks?

>>108581840
>https://github.com/inikep/lzbench has zstd version 1.5.6
so? you linked to lzbench-web
https://github.com/morotti/lzbench-web
>64802d0 8 years ago

>muh lz4
https://github.com/facebook/zstd/releases/tag/v1.3.4
>So far, compression level 1 has been the fastest one available. Starting with v1.3.4, there will be additional choices. >Faster compression levels can be invoked using negative values.
>On the command line, the equivalent one can be triggered using --fast[=#] command.

>>108581868
>so? you linked to lzbench-web
Yes, and it's still a useful tool.
I just did a new computation for you from that 1.5.6 version, and the results are, surprise, not really any different.
>CONFIG_KERNEL_ZSTD uses zstd --ultra 22 so that it can decompress in one pass without allocating buffer.
>Throughput: 1073 MB/s, Compression Ratio: 24.69
>CONFIG_KERNEL_LZ4 uses lz4hc -9.
>Throughput: 3527 MB/s, Compression Ratio: 36.75.
Retarded updooters think somehow their algorithm will double in speed because le updoots.

>So far, compression level 1 has been the fastest one available. Starting with v1.3.4, there will be additional choices. >Faster compression levels can be invoked using negative values.
>On the command line, the equivalent one can be triggered using --fast[=#] command.
In terms of kernel compression, it doesn't matter. The kernel sets it to --ultra -22 with good reason (decompress in-place, one pass).
Go ahead and patch your kernel and let me know how that works out for you.

>>108581909
>Yes, and it's still a useful tool.
see >>108581312
>zstd version 1.3.4
>released this Mar 27, 2018

>muh lz4 see >>108581868
https://github.com/facebook/zstd/releases/tag/v1.3.4
>So far, compression level 1 has been the fastest one available. Starting with v1.3.4, there will be additional choices. >Faster compression levels can be invoked using negative values.
>On the command line, the equivalent one can be triggered using --fast[=#] command.

American education

>>108581909
>Go ahead and patch your kernel and let me know how that works out for you.
https://wiki.archlinux.org/title/Mkinitcpio#COMPRESSION_OPTIONS

>>108581917
See https://github.com/inikep/lzbench
Zstandard fast levels only at best achieve just below 2000 MB/s, and there are LZ4 fast levels that achive just below 5000 MB/s.
Hypothetically, throughput ratio (lz4/zstd) versus kernel goes from 3.5x to 2.5x, ratio of compression ratios goes from 0.67 to 0.78 (zstd/lz4).
LZ4 is almost always better for the use case where you mostly care about decompression speeds, or when I/O speeds are better than a gigabit.

>>108581945
I don't use an initramfs or Archlinux and my PC boots faster than yours.
The math presented applies broadly.
Yes you can improve your boottimes by a few miliseconds if you use LZ4 on an SSD.
If your initramfs is on a slow HDD, then your boottimes might slow down by a few miliseconds using LZ4 versus Zstandard.

>>108582039
dumb retard

>>108582043
Okay Archtranny, continue to defend your Zuck compression.

>>108571933
Literally says STD, it's a virus.

File: img-2026-04-11-15-01-26.jpg (175 KB, 919x1153)

175 KB JPG

>>108582056

>>108574441
>--ultra options which increase ram usage during decompression
Completely irrelevant in practice because it's still only a tiny fraction of the RAM available on any remotely modern computer.
>default of 3 in makepkg.conf
The settings in makepkg.conf only apply to packages you build locally (e.g. AUR), in which case slow compression settings are just a waste of time.

>>108582070
https://github.com/JiaT75/zstd/branches/all

>>108582098
>Completely irrelevant in practice because it's still only a tiny fraction of the RAM available on any remotely modern computer.
Still a problem for embedded, or really using Arch on anything other than a modern 64bit x86 CPU.
Furthermore, you're still going to be NOT on the Pareto Frontier (see >>108579673).
From past experience, the ultra options are similarly far off from xz -9, aka 5-10% larger package sizes.
But go ahead and test it and show me the results.
>The settings in makepkg.conf only apply to packages you build locally (e.g. AUR), in which case slow compression settings are just a waste of time.
I sure hope that package maintainers adjust the setting, but somehow I doubt that when they don't even test their packages.
See >>108574382