Performance issue when a directory contains many files
It's a regression that seems to have been introduced by e5b237d5, but I can't be entirely sure because from this commit and for the next two (translation commits aside), the steps to reproduce the bug below lead thunar to exit in error:
thunar:ERROR:thunar-file.c:4225:thunar_file_unwatch: code should not be reached
Bail out! thunar:ERROR:thunar-file.c:4225:thunar_file_unwatch: code should not be reachedHowever, from the third subsequent commit 6f7c7a3a this error is fixed and the bug is still reproducible, so it's in this area.
Steps to reproduce:
- Prepare the test directory:
mkdir testdir cd testdir mkdir files mkdir empty cd files for ((i=0; i<100000; i++)); do : >file$i; done - Enter
testdir/emptyin thunar
At this point, if you open a task manager in parallel, you can already see that thunar consumes abnormal resources for several seconds before settling down. Now:
- Close thunar
Thunar then enters a seemingly endless loop. I imagine it will end, however, as it does after a while if you create just 10,000 files above, but it looks like quadratic growth or worse here.
The problem is also reproducible if you enter directory testdir/files instead of testdir/empty, but it's all the more remarkable that it happens when you enter testdir/empty, which is empty, but at the same level as testdir/files (at a higher level the bug is no longer reproducible).
Here's a trace when thunar is blocked on closing:
#0 0x00007c0c900e9e5c in g_list_remove (list=0x5566d432fd90 = {...}, data=data@entry=0x5566c955f9f0) at ../glib/glib/glist.c:514
#1 0x00007c0c903753c9 in ip_unmap_sub_dir (sub=sub@entry=0x5566c955f9f0, dir=dir@entry=0x5566c913c790) at ../glib/gio/inotify/inotify-path.c:348
#2 0x00007c0c90375557 in _ip_stop_watching (sub=0x5566c955f9f0) at ../glib/gio/inotify/inotify-path.c:382
#3 _ih_sub_cancel (sub=0x5566c955f9f0) at ../glib/gio/inotify/inotify-helper.c:128
#4 g_inotify_file_monitor_cancel (monitor=0x5566c955f7f0 [GInotifyFileMonitor]) at ../glib/gio/inotify/ginotifyfilemonitor.c:75
#5 0x00007c0c902a37a9 in g_file_monitor_cancel (monitor=0x5566c955f7f0 [GInotifyFileMonitor]) at ../glib/gio/gfilemonitor.c:246
#6 0x00005566c7ba6771 in thunar_file_watch_destroyed (data=0x5566c955f3b0) at thunar-file.c:851
#7 0x00007c0c900cc1e2 in g_data_set_internal (datalist=<optimized out>, key_id=<optimized out>, new_data=<optimized out>, new_destroy_func=<optimized out>, dataset=0x0) at ../glib/glib/gdataset.c:468
#8 0x00005566c7baddf2 in thunar_folder_finalize (object=0x5566c917a040 [ThunarFolder]) at thunar-folder.c:397
#9 0x00007c0c9020072a in g_object_unref (_object=0x5566c917a040) at ../glib/gobject/gobject.c:4459
#10 0x00005566c7bf8b86 in thunar_tree_model_item_reset (item=item@entry=0x5566c918cbb0) at thunar-tree-model.c:1222
#11 0x00005566c7bf921f in thunar_tree_model_item_free (item=0x5566c918cbb0) at thunar-tree-model.c:1203
#12 thunar_tree_model_node_traverse_free (node=<optimized out>, user_data=<optimized out>) at thunar-tree-model.c:1682
#13 0x00007c0c900ef1ee in g_node_traverse_post_order (node=0x5566c90776d0, flags=flags@entry=G_TRAVERSE_ALL, func=func@entry=0x5566c7bf9200 <thunar_tree_model_node_traverse_free>, data=data@entry=0x0)
at ../glib/glib/gnode.c:582
#14 0x00007c0c900ef1c0 in g_node_traverse_post_order (node=0x5566c9077820, flags=flags@entry=G_TRAVERSE_ALL, func=func@entry=0x5566c7bf9200 <thunar_tree_model_node_traverse_free>, data=data@entry=0x0)
at ../glib/glib/gnode.c:572
#15 0x00007c0c900ef1c0 in g_node_traverse_post_order (node=0x5566c8e79470, flags=flags@entry=G_TRAVERSE_ALL, func=func@entry=0x5566c7bf9200 <thunar_tree_model_node_traverse_free>, data=data@entry=0x0)
at ../glib/glib/gnode.c:572
#16 0x00007c0c900ef1c0 in g_node_traverse_post_order (node=0x5566c8e74610, flags=flags@entry=G_TRAVERSE_ALL, func=func@entry=0x5566c7bf9200 <thunar_tree_model_node_traverse_free>, data=data@entry=0x0)
at ../glib/glib/gnode.c:572
#17 0x00007c0c900f7c7f in g_node_traverse
(root=<optimized out>, order=order@entry=G_POST_ORDER, flags=flags@entry=G_TRAVERSE_ALL, depth=depth@entry=-1, func=func@entry=0x5566c7bf9200 <thunar_tree_model_node_traverse_free>, data=data@entry=0x0)
at ../glib/glib/gnode.c:867
#18 0x00005566c7bf8afd in thunar_tree_model_finalize (object=0x5566c8e5f900 [ThunarTreeModel]) at thunar-tree-model.c:393
#19 0x00007c0c9020072a in g_object_unref (_object=0x5566c8e5f900) at ../glib/gobject/gobject.c:4459
#20 0x00005566c7bfd207 in thunar_tree_view_finalize (object=0x5566c8e72b20 [ThunarTreeView]) at thunar-tree-view.c:465
#21 0x00007c0c9020072a in g_object_unref (_object=0x5566c8e72b20) at ../glib/gobject/gobject.c:4459
#22 0x00007c0c908767b7 in gtk_scrolled_window_destroy (widget=0x5566c8e4d6a0 [ThunarTreePane]) at ../gtk/gtk/gtkscrolledwindow.c:2821
#23 0x00007c0c901f1696 in g_closure_invoke (closure=0x5566c8bd2130, return_value=0x0, n_param_values=1, param_values=0x7ffe15ab6370, invocation_hint=0x7ffe15ab62c0) at ../glib/gobject/gclosure.c:834
#24 0x00007c0c90220eb6 in signal_emit_unlocked_R.isra.0
(node=node@entry=0x7ffe15ab6440, detail=detail@entry=0, instance=instance@entry=0x5566c8e4d6a0, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7ffe15ab6370)
at ../glib/gobject/gsignal.c:4008
#25 0x00007c0c902117a2 in signal_emit_valist_unlocked (instance=instance@entry=0x5566c8e4d6a0, signal_id=signal_id@entry=99, detail=detail@entry=0, var_args=var_args@entry=0x7ffe15ab65a0)
at ../glib/gobject/gsignal.c:3520
#26 0x00007c0c902119d7 in g_signal_emit_valist (instance=0x5566c8e4d6a0, signal_id=99, detail=0, var_args=var_args@entry=0x7ffe15ab65a0) at ../glib/gobject/gsignal.c:3263
#27 0x00007c0c90211a94 in g_signal_emit (instance=instance@entry=0x5566c8e4d6a0, signal_id=<optimized out>, detail=detail@entry=0) at ../glib/gobject/gsignal.c:3583
#28 0x00007c0c9094f33e in gtk_widget_dispose (object=0x5566c8e4d6a0 [ThunarTreePane]) at ../gtk/gtk/gtkwidget.c:12166
#29 0x00007c0c902007d5 in g_object_run_dispose (object=0x5566c8e4d6a0 [ThunarTreePane]) at ../glib/gobject/gobject.c:1839
#30 g_object_run_dispose (object=0x5566c8e4d6a0 [ThunarTreePane]) at ../glib/gobject/gobject.c:1829
#31 0x00007c0c906b6900 in gtk_box_forall (container=<optimized out>, include_internals=<optimized out>, callback=0x7c0c9093fbd0 <gtk_widget_destroy>, callback_data=0x0) at ../gtk/gtk/gtkbox.c:2678
#32 0x00007c0c9070775e in gtk_container_destroy (widget=0x5566c8befca0 [GtkBox]) at ../gtk/gtk/gtkcontainer.c:1702
#33 0x00007c0c901f1696 in g_closure_invoke (closure=0x5566c8bd2130, return_value=0x0, n_param_values=1, param_values=0x7ffe15ab68e0, invocation_hint=0x7ffe15ab6830) at ../glib/gobject/gclosure.c:834
#34 0x00007c0c90220eb6 in signal_emit_unlocked_R.isra.0
(node=node@entry=0x7ffe15ab69b0, detail=detail@entry=0, instance=instance@entry=0x5566c8befca0, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7ffe15ab68e0)
at ../glib/gobject/gsignal.c:4008
#35 0x00007c0c902117a2 in signal_emit_valist_unlocked (instance=instance@entry=0x5566c8befca0, signal_id=signal_id@entry=99, detail=detail@entry=0, var_args=var_args@entry=0x7ffe15ab6b10)
at ../glib/gobject/gsignal.c:3520
#36 0x00007c0c902119d7 in g_signal_emit_valist (instance=0x5566c8befca0, signal_id=99, detail=0, var_args=var_args@entry=0x7ffe15ab6b10) at ../glib/gobject/gsignal.c:3263
#37 0x00007c0c90211a94 in g_signal_emit (instance=instance@entry=0x5566c8befca0, signal_id=<optimized out>, detail=detail@entry=0) at ../glib/gobject/gsignal.c:3583
#38 0x00007c0c9094f33e in gtk_widget_dispose (object=0x5566c8befca0 [GtkBox]) at ../gtk/gtk/gtkwidget.c:12166
#39 0x00007c0c902007d5 in g_object_run_dispose (object=0x5566c8befca0 [GtkBox]) at ../glib/gobject/gobject.c:1839
#40 g_object_run_dispose (object=0x5566c8befca0 [GtkBox]) at ../glib/gobject/gobject.c:1829
#41 0x00007c0c9081efa6 in gtk_paned_forall (container=<optimized out>, include_internals=<optimized out>, callback=0x7c0c9093fbd0 <gtk_widget_destroy>, callback_data=0x0) at ../gtk/gtk/gtkpaned.c:2320
#42 0x00007c0c9070775e in gtk_container_destroy (widget=0x5566c8dfc8f0 [GtkPaned]) at ../gtk/gtk/gtkcontainer.c:1702
#43 0x00007c0c901f1696 in g_closure_invoke (closure=0x5566c8bd2130, return_value=0x0, n_param_values=1, param_values=0x7ffe15ab6e40, invocation_hint=0x7ffe15ab6d90) at ../glib/gobject/gclosure.c:834
#44 0x00007c0c90220eb6 in signal_emit_unlocked_R.isra.0
(node=node@entry=0x7ffe15ab6f10, detail=detail@entry=0, instance=instance@entry=0x5566c8dfc8f0, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7ffe15ab6e40)
at ../glib/gobject/gsignal.c:4008
#45 0x00007c0c902117a2 in signal_emit_valist_unlocked (instance=instance@entry=0x5566c8dfc8f0, signal_id=signal_id@entry=99, detail=detail@entry=0, var_args=var_args@entry=0x7ffe15ab7070)
at ../glib/gobject/gsignal.c:3520
#46 0x00007c0c902119d7 in g_signal_emit_valist (instance=0x5566c8dfc8f0, signal_id=99, detail=0, var_args=var_args@entry=0x7ffe15ab7070) at ../glib/gobject/gsignal.c:3263
#47 0x00007c0c90211a94 in g_signal_emit (instance=instance@entry=0x5566c8dfc8f0, signal_id=<optimized out>, detail=detail@entry=0) at ../glib/gobject/gsignal.c:3583
#48 0x00007c0c9094f33e in gtk_widget_dispose (object=0x5566c8dfc8f0 [GtkPaned]) at ../gtk/gtk/gtkwidget.c:12166
#49 0x00007c0c902007d5 in g_object_run_dispose (object=0x5566c8dfc8f0 [GtkPaned]) at ../glib/gobject/gobject.c:1839
#50 g_object_run_dispose (object=0x5566c8dfc8f0 [GtkPaned]) at ../glib/gobject/gobject.c:1829
#51 0x00007c0c907a70e0 in gtk_grid_forall () at ../gtk/gtk/gtkgrid.c:524
#52 0x00007c0c9070775e in gtk_container_destroy (widget=0x5566c8d5db30 [GtkGrid]) at ../gtk/gtk/gtkcontainer.c:1702
#53 0x00007c0c901f1696 in g_closure_invoke (closure=0x5566c8bd2130, return_value=0x0, n_param_values=1, param_values=0x7ffe15ab73a0, invocation_hint=0x7ffe15ab72f0) at ../glib/gobject/gclosure.c:834
#54 0x00007c0c90220eb6 in signal_emit_unlocked_R.isra.0
(node=node@entry=0x7ffe15ab7470, detail=detail@entry=0, instance=instance@entry=0x5566c8d5db30, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7ffe15ab73a0)
at ../glib/gobject/gsignal.c:4008
#55 0x00007c0c902117a2 in signal_emit_valist_unlocked (instance=instance@entry=0x5566c8d5db30, signal_id=signal_id@entry=99, detail=detail@entry=0, var_args=var_args@entry=0x7ffe15ab75d0)
at ../glib/gobject/gsignal.c:3520
#56 0x00007c0c902119d7 in g_signal_emit_valist (instance=0x5566c8d5db30, signal_id=99, detail=0, var_args=var_args@entry=0x7ffe15ab75d0) at ../glib/gobject/gsignal.c:3263
#57 0x00007c0c90211a94 in g_signal_emit (instance=instance@entry=0x5566c8d5db30, signal_id=<optimized out>, detail=detail@entry=0) at ../glib/gobject/gsignal.c:3583
#58 0x00007c0c9094f33e in gtk_widget_dispose (object=0x5566c8d5db30 [GtkGrid]) at ../gtk/gtk/gtkwidget.c:12166
#59 0x00007c0c902007d5 in g_object_run_dispose (object=0x5566c8d5db30 [GtkGrid]) at ../glib/gobject/gobject.c:1839
#60 g_object_run_dispose (object=0x5566c8d5db30 [GtkGrid]) at ../glib/gobject/gobject.c:1829
#61 0x00007c0c9095f79a in gtk_window_forall (container=0x5566c8e29cf0 [ThunarWindow], include_internals=0, callback=0x7c0c9093fbd0 <gtk_widget_destroy>, callback_data=0x0) at ../gtk/gtk/gtkwindow.c:8632
#62 0x00007c0c9070775e in gtk_container_destroy (widget=0x5566c8e29cf0 [ThunarWindow]) at ../gtk/gtk/gtkcontainer.c:1702
#63 0x00007c0c901f1730 in g_closure_invoke (closure=0x5566c8bd2130, return_value=0x0, n_param_values=1, param_values=0x7ffe15ab7920, invocation_hint=0x7ffe15ab7870) at ../glib/gobject/gclosure.c:834
#64 0x00007c0c90220eb6 in signal_emit_unlocked_R.isra.0
(node=node@entry=0x7ffe15ab79f0, detail=detail@entry=0, instance=instance@entry=0x5566c8e29cf0, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7ffe15ab7920)
at ../glib/gobject/gsignal.c:4008
#65 0x00007c0c902117a2 in signal_emit_valist_unlocked (instance=instance@entry=0x5566c8e29cf0, signal_id=signal_id@entry=99, detail=detail@entry=0, var_args=var_args@entry=0x7ffe15ab7b50)
at ../glib/gobject/gsignal.c:3520
#66 0x00007c0c902119d7 in g_signal_emit_valist (instance=0x5566c8e29cf0, signal_id=99, detail=0, var_args=var_args@entry=0x7ffe15ab7b50) at ../glib/gobject/gsignal.c:3263
#67 0x00007c0c90211a94 in g_signal_emit (instance=instance@entry=0x5566c8e29cf0, signal_id=<optimized out>, detail=detail@entry=0) at ../glib/gobject/gsignal.c:3583
#68 0x00007c0c9094f33e in gtk_widget_dispose (object=0x5566c8e29cf0 [ThunarWindow]) at ../gtk/gtk/gtkwidget.c:12166
#69 0x00007c0c9095ae31 in gtk_window_dispose (object=0x5566c8e29cf0 [ThunarWindow]) at ../gtk/gtk/gtkwindow.c:3191
#70 0x00007c0c902007d5 in g_object_run_dispose (object=0x5566c8e29cf0 [ThunarWindow]) at ../glib/gobject/gobject.c:1839
#71 g_object_run_dispose (object=0x5566c8e29cf0 [ThunarWindow]) at ../glib/gobject/gobject.c:1829
#72 0x00007c0c907efc4e in gtk_main_do_event (event=0x5566c8bb8720) at ../gtk/gtk/gtkmain.c:1837
#73 gtk_main_do_event (event=<optimized out>) at ../gtk/gtk/gtkmain.c:1691
#74 0x00007c0c90e5fb77 in _gdk_event_emit (event=0x5566c8bb8720) at ../gtk/gdk/gdkevents.c:73
#75 _gdk_event_emit (event=0x5566c8bb8720) at ../gtk/gdk/gdkevents.c:67
#76 0x00007c0c90eb8438 in gdk_event_source_dispatch.lto_priv () at ../gtk/gdk/x11/gdkeventsource.c:354
#77 0x00007c0c900ed199 in g_main_dispatch (context=0x5566c8b92c20) at ../glib/glib/gmain.c:3344
#78 0x00007c0c9014c3bf in g_main_context_dispatch_unlocked (context=0x5566c8b92c20) at ../glib/glib/gmain.c:4152
#79 g_main_context_iterate_unlocked.isra.0 (context=context@entry=0x5566c8b92c20, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/glib/gmain.c:4217
#80 0x00007c0c900ec712 in g_main_context_iteration (context=context@entry=0x5566c8b92c20, may_block=may_block@entry=1) at ../glib/glib/gmain.c:4282
#81 0x00007c0c90316ed6 in g_application_run (application=application@entry=0x5566c8ba70c0 [ThunarApplication], argc=argc@entry=1, argv=argv@entry=0x7ffe15ab80a8) at ../glib/gio/gapplication.c:2712
#82 0x00005566c7b77f24 in main (argc=1, argv=0x7ffe15ab80a8) at main.c:86Edited by Gaël Bonithon