Dev:2.4/Source/Python/API/Threads

Introduction

The Bad

With the addition of dynamic nodes (pynodes) we have to deal with a new issue when executing Python code inside Blender:

PyNodes are executed during rendering, when the node trees are evaluated. Rendering is multithreaded in Blender and will spawn the number of threads specified in the Threads button, located in the Buttons window -> Scene tab -> Output panel.

When more than one thread was specified, Blender would crash. Obvious reason: we were not ready to run Python in multiple threads.

The Good

To try to make Python code thread-safe in Blender we have to learn about Python's GIL (Global Interpreter Lock) and related API. There are controversial issues related to Python and threads that are worth knowing if you're interested in the language, but we'll skip that here.

Since Python 2.3 the PyGILState_* var and functions were introduced to simplify things and that's what we're using.

What we have to do:

1. Initialize threads support right after initializing Python. That acquires the GIL, so at the end of our init function we had to free it.
2. Wrap code that calls Python C API functions inside PyGILState_Ensure / PyGILState_Release function calls. Hopefully we can be "lazy" here and do it for the main functions, not every BPy API call (check note #2 below)
3. Acquire the lock again before finalizing Python. No need to release it after Python itself is shut down, of course.

• Initialization:

BPY_start_python()
{
   PyThreadState *py_tstate = NULL;
   ...
   Py_Initialize();
   PyEval_InitThreads();
   ...
   py_tstate = PyGILState_GetThisThreadState();
   PyEval_ReleaseThread(py_tstate);
}

• Wrapping calls to Python C API:

function_that_runs_python_code()
{
   PyGILState_STATE gilstate;
   ...
   gilstate = PyGILState_Ensure();

   <ok to call Python API functions here>

   PyGILState_Release(gilstate);
}

• Finalization:

BPY_end_python()
{
   ...
   PyGILState_Ensure();
   Py_Finalize();
}

The Ugly

We've opened a can of worms or maybe even a tiny Pandora's box. Hopefully the current implementation will be enough and all problems will be solved by adding more ensure/release blocks in the few places that we missed or will add in the future, but that's not clear yet.

We will need to test well all possibilities of running Python code in Blender (text scripts, menu scripts, scriptlinks, space handlers, pydrivers, pybutton, pyconstraints, pynodes, missed any?) in all supported platforms.

Hint: if you get a crash in Blender that mentions something about Python, threads and frames (like SIGSEGV in Python's internal threadstate_getframe() function), a pair of PyGILState_Ensure/Release() is probably missing somewhere. Easy to find with a debugger.

Interpreter check interval

By default Python's interpreter releases the GIL every 100 "Python virtual instructions". This will cause race conditions depending both on 1) the amount of code executed in a pynode's __call__ function and 2) the number of rendering threads. This can affect image result and maybe even cause worse problems like crashes. One possible solution is to set this interpreter "check interval" to a much higher value. Maybe even:

import sys
sys.setcheckinterval(sys.maxint)

Or in C:

_Py_CheckInterval = PyInt_GetMax();

For now we're investigating.

From the Python sys module documentation:

setcheckinterval(interval)

Set the interpreter's "check interval". This integer value determines how often the interpreter checks for periodic things such as thread switches and signal handlers. The default is 100, meaning the check is performed every 100 Python virtual instructions. Setting it to a larger value may increase performance for programs using threads. Setting it to a value <= 0 checks every virtual instruction, maximizing responsiveness as well as overhead.

Notes

• Failing to match a gilstate = PyGILState_Ensure() with a PyGILState_Release(gilstate) hangs Blender! Make sure the function where it is called always releases the lock before returning. So if the function has multiple return points after gilstate = PyGILState_Ensure() is called, add PyGILState_Release(gilstate) to all of them or rework the code to guarantee it gets called before exiting.
• Notice that Python API calls and Python code is executed in specific points in Blender: the function to run:
  • Text scripts from the Text Editor (also called when executed from command line)
  • Registered gui, events and button events callbacks
  • File / image selector callback
  • Cleanup functions that run after scripts finish, but call Py API functions
  • Scripts registered in menus
  • Scriptlinks
  • SpaceHandlers
  • PyDrivers
  • PyButton (button evaluation)
  • PyConstraints
  • PyNodes

These are the functions where we need to lock and at the end release the GIL. All the other Python related calls made in Blender are executed in functions called inside them. Crashes may mean we forgot some function that needed the ensure/release block. Hangs can mean we forgot to release the GIL somewhere.

• At specific points, where needed, we can release the lock inside a function using Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS blocks.
• In the finalization function there's no need to grab the return of PyGILState_Ensure() in a variable, since after calling Py_Finalize() The Python Interpreter shuts down.
• At first PyEval_ReleaseLock() was used in our init function to release the GIL after enabling threads. This worked fine on Linux, but hanged on Windows. Thankfully, tstate = PyGILState_GetThisThreadState() + PyEval_ReleaseThread(tstate) seems to work in all our supported platforms.

Background Information

• About threads
• Python C API: Threads
• Embedding Python in Multi-Threaded C/C++ Applications