Partially written files sometimes prematurely have thumbnails generated

Submitted by ykui

Assigned to Xfce Bug Triage

Link to original bug (#16467)

Description

Created attachment 9484 Script for generating 100 images with random noise (to generate thumbnails for)

Sometimes, particularly when many files are saved quickly, having Thunar open in the same directory causes some thumbnails to be generated before the file is finished writing. This results in the thumbnail having a section of grey rows at the bottom. Refreshing or navigating away and back does not cause the thumbnails to be regenerated (for the changed, completely written file).

The issue is somewhat reproducible with the attached script, while having Thunar navigated to the same directory.

Attachment 9484, "Script for generating 100 images with random noise (to generate thumbnails for)":
file_16467.txt

Version: 1.8.12

ykui uploaded an attachment:

Attachment 9485, "Sample of generated images showing some thumbnails having the incomplete grey bar":

ykui said:

May or may not be related https://bugzilla.xfce.org/show_bug.cgi?id=16063

Alexander Schwinn @alexxcons said:

Thanks for reporting !

Are you interested in working on a fix? I can provide guidance, but its unlikely that I will have time to fix this myself for a while.

Help would be much appreciated !

ykui said:

Perhaps I could try, though I do not have high hopes. My main idea is to check the last modified date and delay (timeout? idle?) files that were modified less than a second ago. This would maybe affect large files like videos though (would it be a problem to not have thumbnails for those until they are completed/have gone a short time without being written to?) Perhaps requiring that if a file is not in the cache (already has a thumbnail (?)) that it must wait a second or so, in case the problem is that the partial file completes writing before the thumbnail is generated, so that a new thumbnail is never requested.

The less internals that would have to be touched the better though, dbus does not seem like a healthy past-time.

Alexander Schwinn @alexxcons said:

I would start looking at the thumbnail request sent by thunar. (thunar-thumbnailer.c/.h)

thunar_thumbnailer_queue_files is called in thunar_standard_view_request_thumbnails_real ... possibly some check is required to make sure that the file is "complete". Not sure how a "is complete" check can be done. Possibly there is a md5 checksum or something similar.

You may want to check if there is some promising method in Thuna-file.h .. possibly "thunar_file_get_info" ? Or possibly the realted gfile has that information: https://developer.gnome.org/gio/stable/GFile.html

0001-Bootleg-fix-for-bug-16467-thumnail-generation.patch Managed to log in to gitlab now. I attached this to the bugzilla tracker with some comments. It probably needs some cleaning up and likely changes in how certain things are done (the struct malloc, for instance). The strategy used is to check the time-modified stat of the file, and if it is reasonably recent then call a timer that checks the time periodically, and once enough seconds have passed go forth with the queuing up. Additionally check the modification-date for other user-actions that would have queued it up. There is one minor remaining issue I found which is somewhat rare that is commented on in the attachment at the relevant parts of the code.

(I have to say I highly prefer bugzilla to gitlab.)

Thanks alot for the patch, I will take look for it when I have time !

(I have to say I highly prefer bugzilla to gitlab.)

I like alot beeing able to edit comments to fix my typos. And there is the possibility to use pull requests now, which is very nice .. specially for reviewing bigger patches, like yours :P

Sorry for the long delay!

I can reproduce the bug using your script, and the patch fixes it, thank you !

Now that I am looking more closely at it, I wonder if that is something which should be fixed by the tumbler.

E.g tumbler could probably check mtime, create the thumbnail, check mtime again, and re-create the thumbnail if the checks differ.

Meh, now that I search for it: tumbler#15 (closed)

mentioned in issue tumbler#15 (closed)

added 5. Thumbnails Patch labels

mentioned in issue #669 (closed)

For those who would follow this issue but not tumbler#15 (closed), see tumbler!20 (merged) if you want to test.

mentioned in issue #698 (closed)

mentioned in merge request tumbler!20 (merged)

The situation on modern file-systems with fine-granular modification-timestamps (ext4, exFAT) was improved by tumbler#15 (closed) , though the bug still can happen (e.g. easy to reproduce it on a FAT32 flashdrive)

So I will have closer look on the attached patch (@Meriipu, if you are still motivated, a merge request would be great )

mentioned in issue #38

It has been a while, though if I remember correctly the patch was fairly naive and not-so-clean:

• hardcoded delay of enough seconds since last modification to seemingly not cause issues in my case
• some spaghetti with recursive calls and/or (busy?-)looping to repeatedly perform the "has enough time passed yet"-check
• might have other issues since I am not very familiar with the codebase and some function calls may have gone through indirect or nonconvential paths
• potential issues with long-lasting large downloads that regenerate the thumbnail many times causing a lot of possibly recursive calls
• reading one of my comments it seems it is not a perfect fix, and some adventurous interactions/refreshing still causes partial thumbnails.

overall it feels like a proof of concept rather than something shippable, and I am not sure how I would make it tidier.

The main idea was to if a thumbnail was requested and the mtime was the last few seconds, wait for the mtime to be 5-10 seconds and then request another (to still facilitate thumbnails of downloads-in-progress).

It would be a bit of a struggle to clean it up since I had not seen the codebase before that.