Should Xarray prefer h5netcdf and scipy to netCDF4?

[shoyer]

What is your issue?

#10656 unifies the choose of default engine parameter for reading and writing netCDF files. This is a breaking change, so it's worth considering if we want to bundle in any other related changes.

I think it might make sense to switch the default netCDF engine to chosen from the first valid choice of (h5netcdf, scipy, netCDF4), rather than the current order of (netCDF4, h5netcdf, scipy), because netCDF is generally a bit slower than the other libraries, and plays less nicely with concurrency (because it's written in C rather than Python).

Any thoughts?

[kmuehlbauer]

If we do this switch, we need to think about possible implications.

• users might be unaware they are using h5netcdf/h5py instead of netcdf4-python/netcdf-c (we should have a shout in the docs and docstrings)
• users might relate issues to the wrong library (might also happen now, but we could improve our warning/error output by adding the engine in these messages)
• users might not be able to switch engine back, if hidden in downstream code (would have to be fixed downstream)

To give the user as much freedom as possible, I'd suggest to add an option (nc-engine-order or the like) to set_options with the proposed defaults. The user could change as needed.

[shoyer]

I think an option to set the preferred order of netCDF libraries would be a good intermediate step, whether or not we eventually choose to change the defaults.

…

On Wed, Aug 20, 2025 at 12:16 AM Kai Mühlbauer ***@***.***> wrote: *kmuehlbauer* left a comment (pydata/xarray#10657) <#10657 (comment)> If we do this switch, we need to think about possible implications. - users might be unaware they are using h5netcdf/h5py instead of netcdf4-python/netcdf-c (we should have a shout in the docs and docstrings) - users might relate issues to the wrong library (might also happen now, but we could improve our warning/error output by adding the engine in these messages) - users might not be able to switch engine back, if hidden in downstream code (would have to be fixed downstream) To give the user as much freedom as possible, I'd suggest to add an option (nc-engine-order or the like) to set_options with the proposed defaults. The user could change as needed. — Reply to this email directly, view it on GitHub <#10657 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJJFVXBMPYFLBKQN7AK47D3OQOFPAVCNFSM6AAAAACEJLAEBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMBUGUYTQMJRG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[dcherian]

I agree with both adding an option; and changing the defaults.