Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #4387.
@settings(observability=...)takes three possible values:observability=Trueenables observabilityobservability=Falsedisables observabilityobservability=[option1, option2], likeobservability=["coverage"], enables observability, and also enables whatever options are specified@settings(observability=...)is not set, inherit fromHYPOTHESIS_OBSERVABILITYenvvarHYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVERremoved with no deprecation periodHYPOTHESIS_EXPERIMENTAL_OBSERVABILITYandHYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_CHOICESkept for backwards-compat because it was easy, but are now-undocumented and will be removed at some pointI'd like to keep a very clear separation between
@settings(observability=...), andadd_observability_callback. IMO the former should always write to/be related to.hypothesis/observed, and the latter should never be (ie it's sent in-memory).This brings up a non-trivial issue, probably for a future PR: when adding observability callbacks, there's no way to control the
coverageandchoicesoptions. You don't want to set@settings(observability=["choices"])on the test/profile, because this would write to.hypothesis/observed, and you just want the observations in-memory. Previously this was done withobservability.OBSERVABILITY_COLLECT_COVERAGEandobservability.OBSERVABILITY_CHOICES. I'd like to get rid of those vars eventually, and addadd_observability_callback(options=["coverage"]). But semantic questions like "what if two callbacks where only one sets options=["coverage"] have to be answered. I do think this can be done, I just don't want to do it right now. So I've left those vars in, though now un-documented.[cc @hgoldstein95]
I'd also love to write a hypothesis.works blog post about this, at some point..