@@ -447,7 +447,11 @@ Initializing and finalizing the interpreter
447
447
freed. Some memory allocated by extension modules may not be freed. Some
448
448
extensions may not work properly if their initialization routine is called more
449
449
than once; this can happen if an application calls :c:func:`Py_Initialize` and
450
- :c:func: `Py_FinalizeEx ` more than once.
450
+ :c:func: `Py_FinalizeEx ` more than once. :c:func: `Py_FinalizeEx ` must not be
451
+ called recursively from within itself. Therefore, it must not be called by
452
+ any code that may be run as part of the interpreter shutdown process, such
453
+ as :py:mod: `atexit ` handlers, object finalizers, or any code that may be run
454
+ while flushing the stdout and stderr files.
451
455
452
456
.. audit-event :: cpython._PySys_ClearAuditHooks "" c.Py_FinalizeEx
453
457
@@ -1074,6 +1078,37 @@ thread, where the CPython global runtime was originally initialized.
1074
1078
The only exception is if :c:func:`exec` will be called immediately
1075
1079
after.
1076
1080
1081
+ .. _cautions-regarding-runtime-finalization:
1082
+
1083
+ Cautions regarding runtime finalization
1084
+ ---------------------------------------
1085
+
1086
+ In the late stage of :term:`interpreter shutdown`, after attempting to wait for
1087
+ non-daemon threads to exit (though this can be interrupted by
1088
+ :class:`KeyboardInterrupt`) and running the :mod:`atexit` functions, the runtime
1089
+ is marked as *finalizing*: :c:func:`Py_IsFinalizing` and
1090
+ :func:`sys.is_finalizing` return true. At this point, only the *finalization
1091
+ thread* that initiated finalization (typically the main thread) is allowed to
1092
+ acquire the :term:`GIL`.
1093
+
1094
+ If any thread, other than the finalization thread, attempts to acquire the GIL
1095
+ during finalization, either explicitly via :c:func:`PyGILState_Ensure`,
1096
+ :c:macro:`Py_END_ALLOW_THREADS`, :c:func:`PyEval_AcquireThread`, or
1097
+ :c:func:`PyEval_AcquireLock`, or implicitly when the interpreter attempts to
1098
+ reacquire it after having yielded it, the thread enters **a permanently blocked
1099
+ state** where it remains until the program exits. In most cases this is
1100
+ harmless, but this can result in deadlock if a later stage of finalization
1101
+ attempts to acquire a lock owned by the blocked thread, or otherwise waits on
1102
+ the blocked thread.
1103
+
1104
+ Gross? Yes. This prevents random crashes and/or unexpectedly skipped C++
1105
+ finalizations further up the call stack when such threads were forcibly exited
1106
+ here in CPython 3.13.7 and earlier. The CPython runtime GIL acquiring C APIs
1107
+ have never had any error reporting or handling expectations at GIL acquisition
1108
+ time that would've allowed for graceful exit from this situation. Changing that
1109
+ would require new stable C APIs and rewriting the majority of C code in the
1110
+ CPython ecosystem to use those with error handling.
1111
+
1077
1112
1078
1113
High-level API
1079
1114
--------------
@@ -1147,11 +1182,14 @@ code, or when embedding the Python interpreter:
1147
1182
ensues.
1148
1183
1149
1184
.. note::
1150
- Calling this function from a thread when the runtime is finalizing
1151
- will terminate the thread, even if the thread was not created by Python.
1152
- You can use :c:func:`Py_IsFinalizing` or :func:`sys.is_finalizing` to
1153
- check if the interpreter is in process of being finalized before calling
1154
- this function to avoid unwanted termination.
1185
+ Calling this function from a thread when the runtime is finalizing will
1186
+ hang the thread until the program exits, even if the thread was not
1187
+ created by Python. Refer to
1188
+ :ref:`cautions-regarding-runtime-finalization` for more details.
1189
+
1190
+ .. versionchanged:: next
1191
+ Hangs the current thread, rather than terminating it, if called while the
1192
+ interpreter is finalizing.
1155
1193
1156
1194
.. c:function:: PyThreadState* PyThreadState_Get()
1157
1195
@@ -1207,11 +1245,14 @@ with sub-interpreters:
1207
1245
to call arbitrary Python code. Failure is a fatal error.
1208
1246
1209
1247
.. note::
1210
- Calling this function from a thread when the runtime is finalizing
1211
- will terminate the thread, even if the thread was not created by Python.
1212
- You can use :c:func:`Py_IsFinalizing` or :func:`sys.is_finalizing` to
1213
- check if the interpreter is in process of being finalized before calling
1214
- this function to avoid unwanted termination.
1248
+ Calling this function from a thread when the runtime is finalizing will
1249
+ hang the thread until the program exits, even if the thread was not
1250
+ created by Python. Refer to
1251
+ :ref:`cautions-regarding-runtime-finalization` for more details.
1252
+
1253
+ .. versionchanged:: next
1254
+ Hangs the current thread, rather than terminating it, if called while the
1255
+ interpreter is finalizing.
1215
1256
1216
1257
.. c:function:: void PyGILState_Release(PyGILState_STATE)
1217
1258
@@ -1503,17 +1544,20 @@ All of the following functions must be called after :c:func:`Py_Initialize`.
1503
1544
If this thread already has the lock, deadlock ensues.
1504
1545
1505
1546
.. note::
1506
- Calling this function from a thread when the runtime is finalizing
1507
- will terminate the thread, even if the thread was not created by Python.
1508
- You can use :c:func:`Py_IsFinalizing` or :func:`sys.is_finalizing` to
1509
- check if the interpreter is in process of being finalized before calling
1510
- this function to avoid unwanted termination.
1547
+ Calling this function from a thread when the runtime is finalizing will
1548
+ hang the thread until the program exits, even if the thread was not
1549
+ created by Python. Refer to
1550
+ :ref:`cautions-regarding-runtime-finalization` for more details.
1511
1551
1512
1552
.. versionchanged:: 3.8
1513
1553
Updated to be consistent with :c:func:`PyEval_RestoreThread`,
1514
1554
:c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`,
1515
1555
and terminate the current thread if called while the interpreter is finalizing.
1516
1556
1557
+ .. versionchanged:: next
1558
+ Hangs the current thread, rather than terminating it, if called while the
1559
+ interpreter is finalizing.
1560
+
1517
1561
:c:func:`PyEval_RestoreThread` is a higher-level function which is always
1518
1562
available (even when threads have not been initialized).
1519
1563
0 commit comments