feat: Encode stats for Binary and FixedLengthBinary #3655
Conversation
|
ACTION NEEDED delta-rs follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
HawaiianSpork
force-pushed
the
binanry-stats
branch
2 times, most recently
from
b7f70a0 to
8d731b3
Compare
This now writes binary stats as escaped string sequences according the delta lake protocol instead of logging a warning. The encoding logic was all there, just needed to call it add add tests. Signed-off-by: Michael R. Maletich <[email protected]>
HawaiianSpork
force-pushed
the
binanry-stats
branch
from
8d731b3 to
f2f6b88
Compare
|
@HawaiianSpork when you refer to this being defined in the protocol, are you referring to the partition value serialization specification? I'm not sure what min/max values make sense for binary values 😕 |
|
@rtyler maybe it isn't explicit but I was referring to the protocol says that at least for binary partition identification should be identified by an escaped string https://github.com/delta-io/delta/blob/master/PROTOCOL.md#partition-value-serialization
We would find it useful for filter on hashes. |
Description
This now writes binary stats as escaped string sequences according the delta lake protocol instead of logging a warning. The encoding logic was all there, just needed to call it add add tests.
Related Issue(s)