Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop support for zstandard #7810

Open
crusaderky opened this issue Apr 27, 2023 · 2 comments
Open

Drop support for zstandard #7810

crusaderky opened this issue Apr 27, 2023 · 2 comments

Comments

@crusaderky
Copy link
Collaborator

crusaderky commented Apr 27, 2023

Following A/B tests con Coiled, zstandard seems to be, consistently, a poorer choice than lz4.
I do not see any reason why one would prefer it to lz4 andI think we should drop it in order to reduce our support surface.

A/B tests of zstandard vs. lz4, used for both network comms and spilling, exhibit a 15~80% slowdown in end-to-end runtime across the board:

AB_zstd

XREF #2970
XREF #7403

@fjetter
Copy link
Member

fjetter commented Apr 27, 2023

Is zstd actually causing problems? I don't see the benefits of removing it.

@crusaderky
Copy link
Collaborator Author

Is zstd actually causing problems? I don't see the benefits of removing it.

It isn't; it's just useless. It would be one less moving part to maintain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants