This is triggered by Variety but the deadlock is in xfdesktop (or glib, I will file this there as well)
I traced it down to xfce_backdrop_load_image_files() setting a monitor on the directory the image is in, which takes a lock on the monitor. if there's a inotify event causing a monitor release at the same time the monitor thread grabs the locks as well, deadlocking. I think in this case it's on .xsession-errors being written to?
I hit this quite often because Variety changing the background every few minutes, so every few days the xfdesktop process will lockup.
Backtraces Main thread:
gdb) bt
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x00007faaab03725c in g_mutex_lock_slowpath (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1493
#2 0x00007faaab037d32 in g_mutex_lock (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1517
#3 0x00007faaab274ac4 in _ih_startup () at ../../../gio/inotify/inotify-helper.c:74
#4 0x00007faaab2744b0 in g_inotify_file_monitor_start
(local_monitor=<optimized out>, dirname=0x564029b29890 "/home/USERNAME/.config/variety/Downloaded/Unsplash", basename=0x0, filename=0x0, source=0x56402df58690)
at ../../../gio/inotify/ginotifyfilemonitor.c:61
#5 0x00007faaab269f58 in g_local_file_monitor_new_for_path (pathname=0x56402df725e0 "/home/USERNAME/.config/variety/Downloaded/Unsplash", is_directory=1, flags=G_FILE_MONITOR_NONE, error=0x0)
at ../../../gio/glocalfilemonitor.c:887
#6 0x0000564026070e56 in xfce_backdrop_load_image_files (backdrop=0x5640268bf990 [XfceBackdrop]) at xfce-backdrop.c:536
Monitor thread:
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x00007faaab03725c in g_mutex_lock_slowpath (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1493
#2 0x00007faaab037d32 in g_mutex_lock (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1517
#3 0x00007faaab274bb9 in _ih_sub_cancel (sub=0x56402ae31750) at ../../../gio/inotify/inotify-helper.c:121
#4 0x00007faaab27445a in g_inotify_file_monitor_cancel (monitor=0x7faa9c2221f0 [GInotifyFileMonitor]) at ../../../gio/inotify/ginotifyfilemonitor.c:75
#5 0x00007faaab19ce69 in g_file_monitor_cancel (monitor=0x7faa9c2221f0 [GInotifyFileMonitor]) at ../../../gio/gfilemonitor.c:241
#6 0x00007faaab19ceec in g_file_monitor_dispose (object=0x7faa9c2221f0 [GInotifyFileMonitor]) at ../../../gio/gfilemonitor.c:123
#7 0x00007faaab0e27e1 in g_object_unref (_object=<optimized out>) at ../../../gobject/gobject.c:3636
#8 g_object_unref (_object=0x7faa9c2221f0) at ../../../gobject/gobject.c:3553
#9 0x00007faaab2695f8 in g_file_monitor_source_handle_event
(fms=0x564029abf8d0, event_type=<optimized out>, child=0x7faaa4015980 ".xsession-errors", rename_to=rename_to@entry=0x0, other=<optimized out>, other@entry=0x0, event_time=<optimized out>)
at ../../../gio/glocalfilemonitor.c:455
#10 0x00007faaab2748ae in ih_event_callback (event=0x7faaa4002ac0, sub=0x56402ae31750, file_event=<optimized out>) at ../../../gio/inotify/inotify-helper.c:206
#11 0x00007faaab27509c in ip_event_dispatch (dir_list=dir_list@entry=0x56402df36680 = {...}, file_list=0x0, event=event@entry=0x7faaa4002ac0) at ../../../gio/inotify/inotify-path.c:493
#12 0x00007faaab27528e in ip_event_dispatch (event=0x7faaa4002ac0, file_list=<optimized out>, dir_list=0x56402df36680 = {...}) at ../../../gio/inotify/inotify-path.c:448
#13 ip_event_callback (event=0x7faaa4002ac0) at ../../../gio/inotify/inotify-path.c:548
#14 ip_event_callback (event=0x7faaa4002ac0) at ../../../gio/inotify/inotify-path.c:530
#15 0x00007faaab275d11 in ik_source_dispatch (source=0x5640269dc420, func=0x7faaab2751d0 <ip_event_callback>, user_data=<optimized out>) at ../../../gio/inotify/inotify-kernel.c:327
#16 0x00007faaaafe3e94 in g_main_dispatch (context=0x5640268acae0) at ../../../glib/gmain.c:3417
#17 g_main_context_dispatch (context=0x5640268acae0) at ../../../glib/gmain.c:4135
#18 0x00007faaaafe4238 in g_main_context_iterate (context=context@entry=0x5640268acae0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../../../glib/gmain.c:4211
#19 0x00007faaaafe42ef in g_main_context_iteration (context=0x5640268acae0, may_block=may_block@entry=1) at ../../../glib/gmain.c:4276
#20 0x00007faaaafe4341 in glib_worker_main (data=<optimized out>) at ../../../glib/gmain.c:6178
#21 0x00007faaab00e59d in g_thread_proxy (data=0x564026887e40) at ../../../glib/gthread.c:827
#22 0x00007faaaaf74d80 in start_thread (arg=0x7faaa95a3640) at pthread_create.c:481
#23 0x00007faaaae8e76f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
I just thought to check and there are a few other places that may have the same issue, all have the same pattern of disconnecting but not cancelling the monitor before the _unref.
2 in xfdesktop_file_icon_manager_fini https://gitlab.xfce.org/xfce/xfdesktop/-/blob/master/src/xfdesktop-file-icon-manager.c#L3378
1 in xfdesktop_regular_file_icon_finalize https://gitlab.xfce.org/xfce/xfdesktop/-/blob/master/src/xfdesktop-regular-file-icon.c#L180
and 1 in xfdesktop_special_file_icon_finalize https://gitlab.xfce.org/xfce/xfdesktop/-/blob/master/src/xfdesktop-special-file-icon.c#L157
I haven't seen a lockup due to these, but i never interact with icons on xfdesktop at all, and my Desktop folder doesn't get used. Possibly worth adding the cancel to those just out of caution.
Due to how glib handles file monitors in a separate thread, unreferencing a monitor without canceling first can cause a race and deadlock.
For more details, refer to upstream issue: https://gitlab.gnome.org/GNOME/glib/-/issues/1941
Fixes #188
Signed-off-by: Dan Merillat git@dan.eginity.com
Created a formal MR for it.
Due to how glib handles file monitors in a separate thread, unreferencing a monitor without canceling first can cause a race and deadlock.
For more details, refer to upstream issue: https://gitlab.gnome.org/GNOME/glib/-/issues/1941
Fixes #188
Signed-off-by: Dan Merillat git@dan.eginity.com
It's been three weeks now and 6000+ background changes and I've not seen another lockup. I'm going to say this is properly tested.
Can you apply the one-line fix from the given diff or do you need a formal PR?
It looks like this issue: https://gitlab.gnome.org/GNOME/glib/-/issues/1941
The monitor is actually released in a separate thread so there's a race if you release a monitor and create a new one. The fix is to cancel before unref.
I'm testing this patch now but it might be a week before I'm sure it's fixed it.
diff --git a/src/xfce-backdrop.c b/src/xfce-backdrop.c
index cae5e0bd..a978a025 100644
--- a/src/xfce-backdrop.c
+++ b/src/xfce-backdrop.c
@@ -257,6 +257,7 @@ xfdesktop_backdrop_clear_directory_monitor(XfceBackdrop *backdrop)
G_CALLBACK(cb_xfce_backdrop_image_files_changed),
backdrop);
+ g_file_monitor_cancel(backdrop->priv->monitor);
g_object_unref(backdrop->priv->monitor);
backdrop->priv->monitor = NULL;
}
This is triggered by Variety but the deadlock is in xfdesktop (or glib, I will file this there as well)
I traced it down to xfce_backdrop_load_image_files() setting a monitor on the directory the image is in, which takes a lock on the monitor. if there's a inotify event causing a monitor release at the same time the monitor thread grabs the locks as well, deadlocking. I think in this case it's on .xsession-errors being written to?
I hit this quite often because Variety changing the background every few minutes, so every few days the xfdesktop process will lockup.
Backtraces Main thread:
gdb) bt
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x00007faaab03725c in g_mutex_lock_slowpath (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1493
#2 0x00007faaab037d32 in g_mutex_lock (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1517
#3 0x00007faaab274ac4 in _ih_startup () at ../../../gio/inotify/inotify-helper.c:74
#4 0x00007faaab2744b0 in g_inotify_file_monitor_start
(local_monitor=<optimized out>, dirname=0x564029b29890 "/home/USERNAME/.config/variety/Downloaded/Unsplash", basename=0x0, filename=0x0, source=0x56402df58690)
at ../../../gio/inotify/ginotifyfilemonitor.c:61
#5 0x00007faaab269f58 in g_local_file_monitor_new_for_path (pathname=0x56402df725e0 "/home/USERNAME/.config/variety/Downloaded/Unsplash", is_directory=1, flags=G_FILE_MONITOR_NONE, error=0x0)
at ../../../gio/glocalfilemonitor.c:887
#6 0x0000564026070e56 in xfce_backdrop_load_image_files (backdrop=0x5640268bf990 [XfceBackdrop]) at xfce-backdrop.c:536
Monitor thread:
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x00007faaab03725c in g_mutex_lock_slowpath (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1493
#2 0x00007faaab037d32 in g_mutex_lock (mutex=mutex@entry=0x7faaab3113e8 <g.inotify_lock_lock>) at ../../../glib/gthread-posix.c:1517
#3 0x00007faaab274bb9 in _ih_sub_cancel (sub=0x56402ae31750) at ../../../gio/inotify/inotify-helper.c:121
#4 0x00007faaab27445a in g_inotify_file_monitor_cancel (monitor=0x7faa9c2221f0 [GInotifyFileMonitor]) at ../../../gio/inotify/ginotifyfilemonitor.c:75
#5 0x00007faaab19ce69 in g_file_monitor_cancel (monitor=0x7faa9c2221f0 [GInotifyFileMonitor]) at ../../../gio/gfilemonitor.c:241
#6 0x00007faaab19ceec in g_file_monitor_dispose (object=0x7faa9c2221f0 [GInotifyFileMonitor]) at ../../../gio/gfilemonitor.c:123
#7 0x00007faaab0e27e1 in g_object_unref (_object=<optimized out>) at ../../../gobject/gobject.c:3636
#8 g_object_unref (_object=0x7faa9c2221f0) at ../../../gobject/gobject.c:3553
#9 0x00007faaab2695f8 in g_file_monitor_source_handle_event
(fms=0x564029abf8d0, event_type=<optimized out>, child=0x7faaa4015980 ".xsession-errors", rename_to=rename_to@entry=0x0, other=<optimized out>, other@entry=0x0, event_time=<optimized out>)
at ../../../gio/glocalfilemonitor.c:455
#10 0x00007faaab2748ae in ih_event_callback (event=0x7faaa4002ac0, sub=0x56402ae31750, file_event=<optimized out>) at ../../../gio/inotify/inotify-helper.c:206
#11 0x00007faaab27509c in ip_event_dispatch (dir_list=dir_list@entry=0x56402df36680 = {...}, file_list=0x0, event=event@entry=0x7faaa4002ac0) at ../../../gio/inotify/inotify-path.c:493
#12 0x00007faaab27528e in ip_event_dispatch (event=0x7faaa4002ac0, file_list=<optimized out>, dir_list=0x56402df36680 = {...}) at ../../../gio/inotify/inotify-path.c:448
#13 ip_event_callback (event=0x7faaa4002ac0) at ../../../gio/inotify/inotify-path.c:548
#14 ip_event_callback (event=0x7faaa4002ac0) at ../../../gio/inotify/inotify-path.c:530
#15 0x00007faaab275d11 in ik_source_dispatch (source=0x5640269dc420, func=0x7faaab2751d0 <ip_event_callback>, user_data=<optimized out>) at ../../../gio/inotify/inotify-kernel.c:327
#16 0x00007faaaafe3e94 in g_main_dispatch (context=0x5640268acae0) at ../../../glib/gmain.c:3417
#17 g_main_context_dispatch (context=0x5640268acae0) at ../../../glib/gmain.c:4135
#18 0x00007faaaafe4238 in g_main_context_iterate (context=context@entry=0x5640268acae0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../../../glib/gmain.c:4211
#19 0x00007faaaafe42ef in g_main_context_iteration (context=0x5640268acae0, may_block=may_block@entry=1) at ../../../glib/gmain.c:4276
#20 0x00007faaaafe4341 in glib_worker_main (data=<optimized out>) at ../../../glib/gmain.c:6178
#21 0x00007faaab00e59d in g_thread_proxy (data=0x564026887e40) at ../../../glib/gthread.c:827
#22 0x00007faaaaf74d80 in start_thread (arg=0x7faaa95a3640) at pthread_create.c:481
#23 0x00007faaaae8e76f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95