Display nested attribute dictionaries collapsibly in notebook output

[thomasteisberg]

This change allows nested dictionaries within Dataset attributes to be displayed as collapsible sections in notebook output (_repr_html_).

I'm working on a project that produces xarray Datasets with large nested attribute dictionaries. I couldn't find any way of overriding the _repr_html_ output without subclassing Dataset. We've temporarily implemented an accessor as a work-around, but I'm hoping this change in display format is generally useful enough to make it into xarray.

I'd appreciate any feedback on this display format and if xarray would be interested in incorporating something like this.

Minimal example

import xarray as xr

ds = xr.Dataset(
    {},
    attrs={
        "test": "This is a test dataset",
        "nested_dict": {
            "key1": "value1",
            "key2": "value2",
            "nested": {
                "subkey1": "subvalue1",
                "subkey2": "subvalue2",
            },
        },
        "nested_dict_2": {
            "key3": "value3",
            "key4": "value4",
        },
        "regular_key": "regular_value",
    }
)

ds

Before:

After:

There's an example of how we're using this (currently through an accessor) here: https://www.thomasteisberg.com/xopr/demo-notebook/

[welcome]

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in our contributing guidelines.

[shoyer]

Thanks Thomas for sharing!

My initial thought is that this feels a little complex to add into Xarray for a relatively niche use-case (sorry!). In particular, new CSS gets added to the saved output of every Xarray object, whether it uses dict attributes or not.

One small fix that might help a bit for your use-case is to is clip long attributes in the HTML repr at some max line length, similar to what we already do in the text repr:

<xarray.Dataset> Size: 0B
Dimensions:  ()
Data variables:
    *empty*
Attributes:
    test:           This is a test dataset
    nested_dict:    {'key1': 'value1', 'key2': 'value2', 'nested': {'subkey0'...
    nested_dict_2:  {'key3': 'value3', 'key4': 'value4'}
    regular_key:    regular_value

Then at least Xarray's HTML repr would not take up the entire screen.

[thomasteisberg]

Hi @shoyer thanks for taking a look.

We were trying to come up with the most general form of an improved repr that would still work for us, but it may be that this is still too niche. Given the "do not subclass" recommendation, is there any recommended way to implement a custom _repr_html_ for libraries that build upon Dataset?

I ran across the same question here, but it doesn't seem like it went anywhere.

As far as CSS bloat, this PR adds 702 bytes to an 8048 byte stylesheet. If CSS file size is the main concern, I think it would likely be possible to cut that down more by re-using rules from existing collapsible elements. I'm happy to take a shot at that if it would make a difference, but I suspect the issue here is more about if this functionality is generally useful enough. Please tell me if I'm wrong, of course.

[shoyer]

Thinking about this is a little more, I am coming around to the idea that expandable HTML reprs of attribute values could be broadly useful.

The structure would look something like:

• Create a one line summary of an attribute. This could reuse the _repr_inline_ method, which we currently support for array values, or we could have a specific method for generating one line HTML summaries (maybe _repr_html_inline_?).
• If the summary is shorter than the full repr, add an "expand" icon.
• Clicking on "expand" shows the full repr, reusing protocols like _repr_html_ from IPython notebooks, which Xarray also currently supports for array values (e.g., for showing nice dask.array reprs).

I can see a case for implementing default HTML reprs for built-in Python objects like dict.

There's some related discussion here about nested inline reprs here: #4324