Deadlocks with delegates

[sean-anderson-seco]

Because of the GIL, a python delegate will cause a deadlock if it holds any locks that the main thread tries to acquire, assuming the delegate is called on another thread:

Thread 1                       Thread 2
============================== ================================
PythonEngine.EndAllowThreads()
python_code()
 CSharp.FuncA() 
                               CSharp.ThreadB()
                                lock(obj)
                                CallDelegate()
                                 PythonEngine.EndAllowThreads()
  lock(obj)

Although there are opportunities for deadlock with the explicit locking, one hazardous lock is that used by the garbage collector. Whenever C# code allocates memory, the GC may need to "stop the world" in order to free up resources. If this is done in a cooperative manner (such as with MONO_THREADS_SUSPEND set to coop or hybrid), then deadlock can result. This is because coop mode requires mono threads to periodically execute MONO_SUSPEND_CHECK, which will not occur when PythonEngine.EndAllowThreads is waiting for the GIL. For mono at least, this can be resolved by setting MONO_THREADS_SUSPEND to preemptive, which uses signals to induce GC pauses. As this setting can only be configured with an environmental variable, users will need to set it manually. I am not sure about the behavior of other runtimes.

Of course, it is also possible to introduce deadlocks using explicit locking. Given the delegate-oriented renode API, eliminating the use of python delegates altogether would be unsatisfactory. Ideally, callbacks would not be called with locks held, but sometimes this is necessary to provide API guarantees. I think the best way to approach this would be to document both the general problem (as it can be a bit tricky to recognize), as well as document some specific pitfalls.

Another alternative could be to introduce a select-style interface that could register delegates and actions. However, this would make it difficult to synchronously respond to events. Maybe something using Monitor.TryCompilePlugin could be wired up?