Skip to content

Conversation

@dsprenkels
Copy link
Collaborator

@dsprenkels dsprenkels commented Nov 5, 2025

This PR implements support for 16-bit floating point types. It creates a pf16 (i.e., "polars f16") newtype (to prevent confusion with other f16 definitions). It is based on the f16 type from the half crate, but it is newtype'd such that we can impl an vast amount of traits that we need.

Still TODO

  • Check if all [amber] Check if this code is covered spots are covered by tests
  • Read through all the code once more
  • Check the coverage report to see if there are spots that I do not trust well enough
  • Benchmark the speed and memory usage of some basic operations
  • Update the reference API
  • Update the user guide
  • Add some interop tests
    • Polars IPC <-> pyarrow feather
    • Polars parquet <-> pyarrow parquet
    • Polars arrow <-> pyarrow
    • Polars CSV <-> ?? (maybe just roundtrip)
    • Parquet roundtrip
    • IPC roundtrip
    • JSON roundtrip

Bikeshedding suggestions

  • Is polars-utils the right place to host pf16?

Fixes: #7288
Related: #21803 (not fixed)

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Nov 5, 2025
@github-actions github-actions bot added changes-dsl Do not merge if this label is present and red and removed changes-dsl Do not merge if this label is present and red labels Nov 6, 2025
@github-actions github-actions bot added changes-dsl Do not merge if this label is present and red and removed changes-dsl Do not merge if this label is present and red labels Nov 6, 2025
@github-actions github-actions bot added changes-dsl Do not merge if this label is present and red and removed changes-dsl Do not merge if this label is present and red labels Nov 6, 2025
@dsprenkels dsprenkels marked this pull request as ready for review November 6, 2025 14:46
@dsprenkels dsprenkels marked this pull request as draft November 6, 2025 14:58
@github-actions github-actions bot added changes-dsl Do not merge if this label is present and red and removed changes-dsl Do not merge if this label is present and red labels Nov 6, 2025
@ritchie46
Copy link
Member

Polars CSV <-> ?? (maybe just roundtrip)

We have post-read casts, I think that would be fine. This is done when a user passes a f16 dtype as schema. Round tripping without passing a schema is impossible as we don't encode dtypes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changes-dsl Do not merge if this label is present and red enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Float16 data type

2 participants