Should LWP still be running after Thread.join() returns? - cpython

Question

I have a hung python (CPython 3.6) process. From a gdb backtrace, a forked process waits indefinitely for a mutex held by a LWP/Thread which no longer exists. After some debugging, I can see that after join() returns on the blocking Thread, its LWP is still active, with its stack unwinding with a backtrace like this:
#0 dl_open_worker (a=a#entry=0x7fffefb0eb90) at dl-open.c:515
#1 0x00007ffff7b4b2df in __GI__dl_catch_exception (exception=0x7fffefb0eb70, operate=0x7ffff7de9dc0 <dl_open_worker>, args=0x7fffefb0eb90) at dl-error-skeleton.c:196
#2 0x00007ffff7de97ca in _dl_open (file=0x7ffff77d9bc0 "libgcc_s.so.1", mode=-2147483646, caller_dlopen=0x7ffff77d7deb <pthread_cancel_init+43>, nsid=<optimised out>, argc=9, argv=<optimised out>, env=0x7fffffffe318) at dl-open.c:605
#3 0x00007ffff7b4a3ad in do_dlopen (ptr=ptr#entry=0x7fffefb0edc0) at dl-libc.c:96
#4 0x00007ffff7b4b2df in __GI__dl_catch_exception (exception=exception#entry=0x7fffefb0ed60, operate=operate#entry=0x7ffff7b4a370 <do_dlopen>, args=args#entry=0x7fffefb0edc0) at dl-error-skeleton.c:196
#5 0x00007ffff7b4b36f in __GI__dl_catch_error (objname=objname#entry=0x7fffefb0edb0, errstring=errstring#entry=0x7fffefb0edb8, mallocedp=mallocedp#entry=0x7fffefb0edaf, operate=operate#entry=0x7ffff7b4a370 <do_dlopen>, args=args#entry=0x7fffefb0edc0) at dl-error-skeleton.c:215
#6 0x00007ffff7b4a4d9 in dlerror_run (args=0x7fffefb0edc0, operate=0x7ffff7b4a370 <do_dlopen>) at dl-libc.c:46
#7 __GI___libc_dlopen_mode (name=name#entry=0x7ffff77d9bc0 "libgcc_s.so.1", mode=mode#entry=-2147483646) at dl-libc.c:195
#8 0x00007ffff77d7deb in pthread_cancel_init () at ../sysdeps/nptl/unwind-forcedunwind.c:52
#9 0x00007ffff77d7fd4 in _Unwind_ForcedUnwind (exc=0x7fffefb0fd70, stop=stop#entry=0x7ffff77d5d80 <unwind_stop>, stop_argument=0x7fffefb0ef10) at ../sysdeps/nptl/unwind-forcedunwind.c:126
#10 0x00007ffff77d5f10 in __GI___pthread_unwind (buf=<optimised out>) at unwind.c:121
#11 0x00007ffff77cdae5 in __do_cancel () at pthreadP.h:297
#12 __pthread_exit (value=<optimised out>) at pthread_exit.c:28
#13 0x00007ffff7b14504 in __pthread_exit (retval=<optimised out>) at forward.c:173
#14 0x00000000006383c5 in PyThread_exit_thread () at ../Python/thread_pthread.h:300
#15 0x00000000005e5f0f in t_bootstrap () at ../Modules/_threadmodule.c:1030
#16 0x0000000000638084 in pythread_wrapper (arg=<optimised out>) at ../Python/thread_pthread.h:205
#17 0x00007ffff77cc6db in start_thread (arg=0x7fffefb0f700) at pthread_create.c:463
#18 0x00007ffff7b0588f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Should this LWP be there at all or should it terminate before join()? If the latter, am I looking at a CPython bug?

Answer 1

From inspection of the source code, it seems that this is expected. From https://github.com/python/cpython/blob/master/Modules/_threadmodule.c#L1028, when a thread has finished its work, these two calls are made:
PyThreadState_DeleteCurrent();
PyThread_exit_thread();
The first ends up releasing the lock which allows join() to return; the second results in the LWP unwinding its stack.
Debugging a trivial test python script which starts a Thread, join()s it and then sends a signal to itself (which provides a convenient breakpoint which gdb can understand) shows a similar backtrace to the one in the original post, which further supports this conclusion.