Skip to content

Conversation

sgreenbury
Copy link
Collaborator

@sgreenbury sgreenbury commented Oct 7, 2025

Closes #720.

This PR adds support for including emulator uncertainty in the confidence intervals for sensitivity analysis.

Approach

See description in #720 and end of section 3.2 for decomposing variance into SA sampling uncertainty and emulator uncertainty.

The approach extends the confidence intervals provided by SALib by adding an additional orthogonal variance as a result of bootstrap samples of the predictive distriibution from the emulator with the SA run and output recorded for each bootstrap sample to generate a distribution of SA outputs with respect to the emulator uncertainty.

An alternative could be to directly build the predictive sampling of outputs into a single bootstrap sampling approach that is currently within SALib for capturing the uncertainty of the SA estimates due to the input sampling.

API design choice

A design choice that might be good to consider here: currently this impl leaves the including emulator uncertainty as optional and when set to True with a non-UQ emulator, raises an error. This is motivated by:

  • Enabling the current SA behaviour to be retained through setting this flag as False and opting in to additional expensive bootstraps
  • Enabling maximum flexibility

@sgreenbury sgreenbury force-pushed the 720-emulator-uq-for-sa branch from 635b3a8 to feee37e Compare October 7, 2025 17:23
This enables the predicted distribution to be sampled
from directly when constructing bootstrap samples
and can therefore capture covariance if the predicted
distribution supports this.
@sgreenbury sgreenbury marked this pull request as ready for review October 8, 2025 16:37
@radka-j
Copy link
Member

radka-j commented Oct 9, 2025

This is great, thank you! I will review properly later today but wanted to mention that we might want to add this feature to our docs. Happy for this to be a separate PR (see #880).

@sgreenbury sgreenbury changed the title Add support for emulator UQ to be included in sensitvity analysis Add support for emulator UQ to be included in sensitivity analysis Oct 9, 2025
@radka-j radka-j self-requested a review October 9, 2025 13:13
Copy link
Member

@radka-j radka-j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks!

Overall, I think this is a reasonable approximation. In the case that the two variances are not entirely independent, we might be slightly over-estimating the credible interval but I suspect not by much. And doing some kind of bootstrapping where we estimate the SA sampling error for each GP posterior draw seems like a computational overkill.

Co-authored-by: Radka Jersakova <[email protected]>
@sgreenbury sgreenbury merged commit c20653e into main Oct 15, 2025
5 checks passed
@sgreenbury sgreenbury deleted the 720-emulator-uq-for-sa branch October 15, 2025 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sobol Index Calculation - Use Emulator Uncertainty if available

2 participants