Occasionally `CFRunLoopStop` Does Not Stop the Run Loop on Linux

[gerickson]

When hitting control-C (generating SIGINT) and then issuing CFRunLoopStop in the signal handler, the run loop frequently does not stop on Linux. This is contrast to macOS 10.15.7 in which it stops 100% of the time, as expected.

[gerickson]

Looking at a test program, hlxc from openhlk, the main thread appears to be deadlocked on the spin lock in CFRunLoop.c at line 483:

  Id   Target Id         Frame 
* 1    Thread 0x7faa9cea2bc0 (LWP 34268) "hlxc" pthread_spin_lock () at ../sysdeps/x86_64/nptl/pthread_spin_lock.S:32
  2    Thread 0x7faa9cec3700 (LWP 34282) "hlxc" pthread_spin_lock () at ../sysdeps/x86_64/nptl/pthread_spin_lock.S:32
(gdb) bt
#0  pthread_spin_lock () at ../sysdeps/x86_64/nptl/pthread_spin_lock.S:32
#1  0x00007faa9c1eeb45 in __CFSpinLock (lockp=0x13fae50) at ../CFInternal.h:485
#2  __CFRunLoopLock (rl=0x13fae40) at ../CFRunLoop.c:483
#3  CFRunLoopStop (rl=0x13fae40) at ../CFRunLoop.c:2221
#4  0x0000000000405203 in Client::Stop (this=<optimized out>, aStatus=<optimized out>) at ../../../src/hlxc/hlxc.cpp:573
#5  0x0000000000407e94 in OnSignal (aSignal=<optimized out>) at ../../../src/hlxc/hlxc.cpp:1097
#6  <signal handler called>
#7  0x00007fff24ce174a in ?? ()
#8  0x00007fff24ce1982 in clock_gettime ()
#9  0x00007faa9b14ad36 in __GI___clock_gettime (clock_id=clock_id@entry=1, tp=tp@entry=0x7fff24c7b280) at ../sysdeps/unix/clock_gettime.c:115
#10 0x00007faa9c1f1adf in __CFReadTSR () at ../ForFoundationOnly.h:513
#11 __CFRunLoopRun (waitIfEmpty=0 '\000', stopAfterHandle=<optimized out>, seconds=<optimized out>, rlm=<optimized out>, rl=0x13fae40)
    at ../CFRunLoop.c:2073
#12 CFRunLoopRunSpecific (rl=0x13fae40, modeName=<optimized out>, seconds=<optimized out>, returnAfterSourceHandled=<optimized out>)
    at ../CFRunLoop.c:2118
#13 0x00007faa9c1f200a in CFRunLoopRun () at ../CFRunLoop.c:2132
#14 0x0000000000407be8 in main (argc=<optimized out>, argv=0x7fff24c7ba68) at ../../../src/hlxc/hlxc.cpp:2368

For the second thread:

[Switching to thread 2 (Thread 0x7faa9cec3700 (LWP 34282))]
#0  pthread_spin_lock () at ../sysdeps/x86_64/nptl/pthread_spin_lock.S:32
32	in ../sysdeps/x86_64/nptl/pthread_spin_lock.S
(gdb) info threads
  Id   Target Id         Frame 
  1    Thread 0x7faa9cea2bc0 (LWP 34268) "hlxc" pthread_spin_lock () at ../sysdeps/x86_64/nptl/pthread_spin_lock.S:32
* 2    Thread 0x7faa9cec3700 (LWP 34282) "hlxc" pthread_spin_lock () at ../sysdeps/x86_64/nptl/pthread_spin_lock.S:32
(gdb) bt
#0  pthread_spin_lock () at ../sysdeps/x86_64/nptl/pthread_spin_lock.S:32
#1  0x00007faa9c1eeb45 in __CFSpinLock (lockp=0x13fae50) at ../CFInternal.h:485
#2  __CFRunLoopLock (rl=0x13fae40) at ../CFRunLoop.c:483
#3  CFRunLoopStop (rl=0x13fae40) at ../CFRunLoop.c:2221
#4  0x0000000000405203 in Client::Stop (this=<optimized out>, aStatus=<optimized out>) at ../../../src/hlxc/hlxc.cpp:573
#5  0x0000000000407e94 in OnSignal (aSignal=<optimized out>) at ../../../src/hlxc/hlxc.cpp:1097
#6  <signal handler called>
#7  0x00007fff24ce174a in ?? ()
#8  0x00007fff24ce18c8 in gettimeofday ()
#9  0x00007faa9c1d687d in __CFFileDescriptorManager (arg=<optimized out>) at ../CFFileDescriptor.c:599
#10 0x00007faa9b4126db in start_thread (arg=0x7faa9cec3700) at pthread_create.c:463
#11 0x00007faa9b13b71f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

[gerickson]

This seems substantially improved with #86; however, this still occasionally happens in applications that use OpenCFLite-635.