-
Notifications
You must be signed in to change notification settings - Fork 627
Open
Labels
enhancementit's not broken, but we want it to be betterit's not broken, but we want it to be betterinterophow to play nicely with other packageshow to play nicely with other packages
Description
Based on this mailing-list question and discussion in #4381, I think we have an option to include the choice sequence and spans in observability outputs (respectively as metadata.choices and metadata.choice_spans).
- The choice sequence can mostly be represented directly in JSON, plus
{"nan": <integer value>}and{"bytes": <base64 value>}. And{"integer", <string value>}for large integers... - We might want to store the choice nodes though, which have the constraints on each choice? Hmm.
metadata.choice_nodesif so. - Spans also seem useful, though it's not entirely clear how to store them - maybe a dict with keys label/start/end, and lists of integers?
- We should probably also ship functions to transform these into various forms though, including (choice_sequence, spans) -> labelled_tree.
Finally, I'm inclined to say that this means observability is graduating from experimental to a normal supported part of Hypothesis; and we should therefore think a bit about how it's configured. We'll have options (none, basic, coverage, choices, all); might we end up adding more? If not, it's easy to make that a setting and HYPOTHESIS_OBSERVABILITY= env var.
Metadata
Metadata
Assignees
Labels
enhancementit's not broken, but we want it to be betterit's not broken, but we want it to be betterinterophow to play nicely with other packageshow to play nicely with other packages