Skip to content

Convergence failure of minimize function in power upper limit #945

@yashbhargava1992

Description

@yashbhargava1992

Description of the Bug

The minimize function in the power upper limit uses the default method for convergence in scipy.optimize.minimize. For some test cases, the code seems to fail to reach convergence and therefore it is better to freeze the method. As a test case I also tried using "Nelder-Mead" algorithm and it seems to result in acceptable minimization.

Steps/Code to Replicate the Bug

from scipy.stats import chi2  
import matplotlib.pyplot as plt
from stingray.stats import power_upper_limit

<img width="550" height="413" alt="Image" src="https://github.com/user-attachments/assets/6951cb67-e7fa-450d-9e3a-0f6295013fb0" />

n_avg = 2132 # Number of PDS being averaged. 
df=n_avg*2
pds = chi2.rvs(df, size=2048)/n_avg # size is the number of fourier frequencies

left_conf = 1e-5
right_conf = 1-left_conf
x = np.linspace(chi2.ppf(left_conf, df),
                chi2.ppf(right_conf, df), 100)
pdf = chi2.pdf(x, df)

plt.plot(x/n_avg, pdf*n_avg, 'r-', lw=5)
_ = plt.hist(pds, 100, density=True)

ul_997 = power_upper_limit(np.max(pds), n_avg, 0.997)

plt.axvline(ul_997, ls='--', color='C2')

Expected Results

Input parameters for power_upper_limit: pmeas=2.1490202500317634, n=2132, c=0.997

9301.852278588847 8401.283872079606 9301.852278588847
message: Optimization terminated successfully.
success: True
status: 0
fun: 3886.4796001657623
x: [ 4.651e+03]
nit: 5
nfev: 10
final_simplex: (array([[ 4.651e+03],
[ 4.651e+03]]), array([ 3.886e+03, 3.886e+03]))

Actual Results

pmeas:2.129734870073109, n:2132, c:0.997

message: CONVERGENCE: RELATIVE REDUCTION OF F <= FACTR*EPSMCH
success: True
status: 0
fun: 1.0177245712839067e-09
x: [ 5.567e+02]
nit: 8
jac: [ 7.441e-01]
nfev: 136
njev: 68
hess_inv: <1x1 LbfgsInvHessProduct with dtype=float64>

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIt's a bug! Unexpected or unwanted behavior.help wantedWe need additional help with these issues!

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions