Thunar becomes extremely slow/unresponsive when working in directories with many images

Just tried with 12k random noise png files, 400x100 pixels. So far I did not see a slowdown (thumbnaiing enabled, tried with and w/o folder specific settings, different views).

When increasing the zoom level (switch to large x-large thumnails) I see the spinner for few seconds and have to wait. Though considering the number of files, I suppose that can be expected. At least, I would not describe it as slow/unresponsive.

Though your steps to reproduce are rather vague ... possibly I am missing some detail.

Platforms tested: Thunar 4.16 and 4.18 in Xubuntu and Fedora. Didn't notice a major difference in behavior between versions.

Which versions exactly ? E.g. between the 4.18.x point-releases several thumbnailing fixes were applied.

Steps to reproduce: Best way to recreate this scenario is take a video and dump the frames into a new folder ffmpeg -i video.mkv newdir/%d.png , then open the folder in Thunar and change the view to icons. I aim for around 10k images in testing. Average size of image or if they have already been thumbnailed or not does not seem to matter.

Using a video IMO sounds rather unnecessary and complex for a reproducer. Can you as well reproduce the issue with copying one of the *.png files very often ? Best just attach one of these images to this issue.

So required details:

minimal *.png image to reproduce (and ideally a script to multiply it)
Does it as well happen when thumbnailing is disabled ?
Does it as well happen if folder-specific-settings is disabled ?
Do you maybe use 'draw frames around thumbnails' ?
I tested this on a ssd (Locally I don't have a HDD for testing) ... you too ?

added 1. Performance 3. Need Info labels

Some good points! ffmpeg is my hammer, apologies.

I went back and tested things in 4.18.4, as that's what is in Fedora 38 Beta. This is the 4.18 environment mentioned before, I just took better notes this time. I guess my perception was off because it was difficult to get the "this has been frozen for 30 seconds now" latency, either here or on older versions. It's slow, and it takes a LOT to get there,and it doesn't seem to be disk IO related. Breaking all of that down:

Thunar 4.18.4 on Fedora 38 Beta. Previously experienced this issue on 4.16.10.
Used two test files. One is a 4K desktop JPG export of the "teal" XFCE art, included with Fedora. Second was just a 200x300 or so cropped PNG screenshot of the desktop. I'm only including the former as an attachment.
Generated more files with for i in {1..10000}; do cp 0.jpg $i.jpg; done. Adjust to taste.
Testing was done on a Intel P1600X drive. Speedy SSD. Previously experienced on a different, slower SSD.

As for results:

Things were a little sluggish with 5000 images, but only slightly. 10,000 a bit worse, 20,000 a bit worse still.
Enabling or disabling thumbnails (including removing the thumbnail cache where applicable) did not make a noticeable difference
Toggling the "draw frames around thumbnails" setting did not change the responsiveness, but toggling that setting did produce a similar behavior to the worst case scenarios
Long filenames (32-40ish characters) DID make things noticeably worse. for i in {1..10000}; do cp 0.jpg FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF$i.jpg; done
The 200x300 file set seemed to create less issues than the 4k file set, even with thumbnails turned off.

So with the 4k wallpaper images, 40 character filenames, and 20k files, changing the view could take around 12 seconds. It was inconsistent and usually took at least a few seconds, sometimes about a dozen. This would also, but not exclusively, happen on the very large icons. And initially loading the folder.

But the big takeaway is either I can't reproduce the behavior as bad as I thought it was ("Dozens of seconds" @ 10k), so either I'm missing something or I'm full of it. It does get a bit slow, and opening the folder in Thunar had a similar delay too. The bit that is off for me is that it appears to be something in the software not being able to keep up, there should be enough disk IO and CPU to load the folder quickly even if it does have an obnoxious file count in it with obnoxiously long file names.

Hmm, just tried with 20k copies of your image, using your script to get the desired file-names. Still only ~4seconds here for switching between views, but ~10sec for changing zoom levels.

I suppose the reason for this issue is already described in #528, #937 (closed) or #918 ... let's best wait until some of these are fixed, and then re-test (Currently I don't have time to do so ... help would be very welcome!)

Ah, I've not been accurate. Changing zoom levels is what has been the big lag inducer for me. Changing views not so much. Opening folders can also be delayed, though I have not done extensive testing on that with thumbnails turned off.

It sounds like those issues you mentioned may very well be related. I'm still convinced this isn't IO limited, perhaps also not IO related, but I don't have the means to verify that past "I don't see a bunch of disk activity in nmon while testing". And the fact that it didn't improve at all on very fast (latency wise) storage.

Sounds like that's about all I can add. Hopefully one of the more detailed reports like that will help get this fixed in the future. :)

removed 3. Need Info label

I opened /usr/bin which contains 6843 files. It opens very fast (much less than 1 sec), but refreshing takes 3 seconds. When I select all files (Ctrl-A) and refresh, it takes ~35 seconds.

Also scrolling is very slow when there are selected files (12 FPS only according to recorded video).

Yea, I can confirm that Ctrl-A + refresh takes ages, no matter which view is used.

It looks like the refresh somehow temporary resets the used zoom-level.

That might be a good reproducer. However, I think it makes sense to first fix the general faults which were identified in the issues above. If that got fixed, we can re-test this one.

Also scrolling is very slow when there are selected files (12 FPS only according to recorded video).

A bit notchy for me when thumbnailing is enabled, though without thumbnailing it is smooth for me.

A bit notchy for me when thumbnailing is enabled, though without thumbnailing it is smooth for me.

Hmm, /usr/bin doesn't have thumbnails. I have 3840x2160 screen size and maximized Thunar. When window is small, it's smooth. Files must be selected when scrolling.

Also scrolling is very slow when there are selected files (12 FPS only according to recorded video).

Looks like gtk_cell_renderer_render with flags (GTK_CELL_RENDERER_SELECTED) is very slow (called in exo_icon_view_paint_item).

I think the slowness from selection happens due to the inherent data structure / algorithm deployed to fetch/update the changes in the selection.

Every time "selection changed" signal is called, in thunar_standard_view_selection_changed of thunar-standard-view.c the current list of selected gtk_tree_model_paths's are fetched from the view and this path GList is iterated over to get underlying ThunarFile for each path. This takes O(n) time for each time the selection changes. Now if the selection changed is called for all the n files then this will take almost about O(n^2) time. This conversion of paths to files is I feel the bottleneck and cause for a lot of issues. @alexxcons and I had previously discussed throttling the "selection_changed" signal.

Additionally the "selection changed" signal is connected to callback that fetches something from Xfconf somewhere. Might this be limiting factor as well?

@alexxcons and I had previously discussed throttling the "selection_changed" signal.

I suppose it would be worth a try. Even if it is not the bottleneck for this one, I think it is a rather bad idea to send 'selection changed' signals in high rates,

I opened /usr/bin which contains 6843 files. It opens very fast (much less than 1 sec), but refreshing takes 3 seconds. When I select all files (Ctrl-A) and refresh, it takes ~35 seconds.

Just checked: I can still reproduce the issue.

I have added a small 10 ms delay to selection change using a timeout. Does this patch solve the refresh issue in /usr/bin ?

Very nice @Elessar1802 , that indeed fixes it. \o/ Actually, I wonder why I did not think about g_timeout_add before

The patch needs some minor polishing before pushing it (the new #define is unused, g_source_remove in finalize, style) Besides that, looks good to go.

mentioned in merge request !372 (closed)

added Patch label

mentioned in commit c882a328

Meanwhile, the bug anyhow is mitigated by throttling as well the "selection-changed" signal of the folder-monitor (!429 (merged)). Nevertheless, I think it makes sense to as well throttle selection changed, like suggested by the patch.

So I finally pushed it ... sorry @Elessar1802 , no idea what took me so long for this one

closed

changed milestone to %4.19.3

This change as well seems to fix #114 (closed)

mentioned in issue #114 (closed)

mentioned in commit c54e2674

mentioned in issue #341 (closed)

mentioned in issue #1429 (closed)

Thunar becomes extremely slow/unresponsive when working in directories with many images

Designs

Child items ...

Activity