When writing this package I implicitly assumed that that whenever the log density is finite, the gradient (and now the Hessian, see #101) are also.

So calling eg logdensity_and_gradient in the context of an MCMC sampler

fx, Dfx = logdensity_and_gradient(f, x)
if isfinite(fx)
   # proceed using DfX
else
   # reject point
end

The motivation for the non-finite log density is to provide an escape hatch for x being outside the support, non-convergent solvers, etc.

Should we

1. document that whenever the log density is finite, so are the gradient and the Hessian?

2. or allow cases of finite log density, with potentially non-finite gradient and Hessian? (What would be the use case?)