Support OpenZL compression for Parquet #17950
vadimpiven
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Meta just published their new compression framework OpenZL https://engineering.fb.com/2025/10/06/developer-tools/openzl-open-source-format-aware-compression-framework/ which provides very good compression for parquet (check figure 3). There is even an instruction on applying compression to parquet https://github.com/facebook/openzl/blob/d177e9f32bd5ac037d54a2af5f7588b841aeb47d/doc/mkdocs/doc/getting-started/examples/cli/parquet.md May be DataFusion can support this way of compression?
Beta Was this translation helpful? Give feedback.
All reactions