Support queued copy (configurable e.g. for remote devices)

Cyrille Pontvieux @jrd submitted a patch:

~~Patch 9718~~, "patch based on rename-on-fail-copy branch":
8ab28d41396f3f01e7e7582e6a803925875deb56.patch

Cyrille Pontvieux @jrd said:

gtk-media-play-ltr and gtk-media-pause icon names seems to be ok. I found them in icons/gnome and they symlinks to media-playback-start and media-playback-pause respectively. Them or their symlinks could potentially be a good replacement.

Is there a way to find out a somehow standard, or de-facto standard, list of icons expected on a regular Linux or BSD system?

Cyrille Pontvieux @jrd said:

Sorry my last comment was on wrong bugreport :-(

Reuben Green @reubengreen73 said:

So this patch fails on my system.

The attached patched is based on my previous patch/bug (https://bugzilla.xfce.org/show_bug.cgi?id=16686) I have applied the patches from bugs 16685 and 16686 on top of the latest master (with "git am"), but running "git am" to apply this patch results in a failure:

error: patch failed: thunar/thunar-progress-view.c:622 error: thunar/thunar-progress-view.c: patch does not apply

Maybe there has been some change on master in the meantime? Or maybe I am doing something dumb :)

Cyrille Pontvieux @jrd submitted a patch:

~~Patch 9730~~, "New patch to apply against pause-on-copy patch only.":
0001-Freeze-new-copy-where-source-or-target-device-is-sha.patch

Reuben Green @reubengreen73 said:

So this now applies, compiles and runs fine on my system. Thanks!

One possible bug: if I copy one file onto a device and then while the copy is running start another file copy, I see both jobs in the File Operations dialog with the first job running and the second one frozen with a "Force" button. Now I would expect that if I press the "Force" button, it should turn into a "Pause" button so that I can pause/resume the second job like I can with the first job. However, if I do press the "Force" button, it just stays as a "Force" button, and pressing it again has no effect, so that there is no way to pause the second job. If I let the first job complete then the second job starts and the "Force" button changes to a "Pause" button as expected.

Reuben Green @reubengreen73 said:

Also Cyrille, is this related to bug #12123?

Cyrille Pontvieux @jrd said:

Ah, this worked at some point, so I must have made a last-minute change that breaks it. I will investigate.

About the bug #12123, it's an other way of achieving the same goal. I did a proposal some years ago on this bug, and so I think the features are related even if resolved in another way.

Cyrille Pontvieux @jrd submitted a patch:

Fixed and some modification also applied regarding coding standards.

Patch 9743, "Patch against latest master":
0001-Freeze-new-copy-where-source-or-target-device-is-sha.patch

Reuben Green @reubengreen73 said:

(In reply to Cyrille Pontvieux from comment 9)

Created attachment 9743 [details] Patch against latest master

Fixed and some modification also applied regarding coding standards.

Yep, this fixes the bug from comment 6. Thanks for the fix, Cyrille!

Cyrille Pontvieux @jrd said:

!1 (merged)

Alexander Schwinn @alexxcons closed a related bug:

*** Bug 12577 has been marked as a duplicate of this bug. ***

Alexander Schwinn @alexxcons said:

I just played a bit with the new feature, very nice !

IMO it fixes what was requested in Bug #12123 / is a duplicate, though I dont want to close it as a dup. since there is a bounty of 100$ on it: https://www.bountysource.com/issues/25667178-feature-request-queued-file-transfer ... after pushing it, it's up to Beron to decide if Bug #12123 got fixed by it / you get that reward :)

I am a bit unhappy with the preferences: If I copy some files from my ssd to my ssd, I would like to have parallel copy as default --> untick both If I copy some big files from my flashdrive to my ssd, I would need t tick "one transfer per source device" If I copy some big files from my ssd to my flashdrive, I would need t tick "one transfer per target device"

So I would need to toggle these bits very often, whenever I copy many files to/from my USB drive. That's not very comfortable.

The author of Bug #12577 mentioned that he would like to have queued file transfer only for block devices ... while I am not so sure what exactly is a block-device, and what is not, I can get his point.

Possibly we could use "thunar_file_is_local" to identify if it makes sense to enable queued mode. We could have a nice preferences setting with it by using an enum:

Transfer files in parallel: [ Always | only for local files | Never ] ... actually the same like for "show thumbnails". You probably can copy the code.

You think that would make sense ? Or possibly there is some gain to separate sources from targets, which I dont get ?

Cyrille Pontvieux @jrd said:

Why would you want parallel copies from your ssd to your ssd? I don't see the point, as it will slow down a bit the whole copies. And if you really want, you could hit the Force button. But maybe I don't understand something here.

About having two separate options in the parameters dialog for source and destination, I'm really not sure too. Maybe one option would be enough.

I can also maybe improve the code with using thunar_file_is_local but I need to be sure what a local file is for thunar. I will investigate the thumbnails code to see if it could indeed be of any help here.

For you're case (and probably for my personal laptop too), the « Only for local files » would be ok as a SSD is usally quick enough. But for my work laptop (mechanical disk), I will probably choose « Never ». And « Always » would be like the actual behavior.

I'll inform here when the branch will be updated accordingly (if technically possible).

Cyrille Pontvieux @jrd said:

Well it's not that simple:

thunar_file_is_local simply checks whether the scheme is file:// or not. It just excludes any http://, and the like, schemes.
G_FILE_ATTRIBUTE_FILESYSTEM_USE_PREVIEW is queried over the filesystem. But from what I checked with the thumbnail setting, it still builds thumbnails for any images I have on my USB key, despite I checked « only for local files ». That means that G_FILESYSTEM_PREVIEW_TYPE_IF_LOCAL is probably returned for my USB key even if it's not really « local ».

So I'm not sure that any of these settings will apply correctly to copy in sequences when using a slow USB key/disk. It will probably work for copying over the network (like sshfs, nfs, …).

The checks for the device id still seams a better option.

The only drawback seems when doing multiple copying from and to a fast disk, like an internal SSD. One way to fix this would be to allow parallel copies when source and destination device ids are the same. Not sure how to display this as an understandable option…

By the way, I think that the thumbnail option might benefit from any finding here for what is really a « local file ».

Cyrille Pontvieux @jrd said:

Using g-file-find-enclosing-mount and then g-mount-can-eject/g-mount-can-unmount could be an option to find out really not local filesystems (when advised as local by thunar_file_is_local) and hence probably slow device access.

Cyrille Pontvieux @jrd said:

Ok I made some tests (fuse mounts, USB disk mounts, PTP mounts, MTP mounts, Samba mounts, and local files on different partitions).

g_file_find_enclosing_mount returns a non-NULL gmount only for attached devices ; for local files it returns NULL. Then a test on g_mount_can_unmount could be done. Of course thunar_file_is_local should be done first.

I think by mixing all of these, the UX could be satisfactory. I will come with a new commit today or tomorrow with this idea.

Alexander Schwinn @alexxcons said:

True, for my flash drives, as well thumbnails are generated :/ ...jepp, thumbnail settings could possibly profit from your planned changes :)

(In reply to Cyrille Pontvieux from comment 15)

One way to fix this would be to allow parallel copies when source and destination device ids are the same.

Sounds like a good idea to me. I guess if there are multiple hdds/ssds, it would be ok to have queued file-transfer

(In reply to Cyrille Pontvieux from comment 17)

Ok I made some tests (fuse mounts, USB disk mounts, PTP mounts, MTP mounts, Samba mounts, and local files on different partitions).

g_file_find_enclosing_mount returns a non-NULL gmount only for attached devices ; for local files it returns NULL. Then a test on g_mount_can_unmount could be done. Of course thunar_file_is_local should be done first.

I think by mixing all of these, the UX could be satisfactory. I will come with a new commit today or tomorrow with this idea.

Promising !, thanks alot !

In case you want to as well check for "same device id", we probably would need an additional enum value .. e.g: Transfer files in parallel: [ Always | only for local files | only for local files if source device matches target device | Never ] ... hope that text would not be too long.

Cyrille Pontvieux @jrd said:

Ok this is now done and tested (on my part).

I pushed another commit and rebased on master.

Please tell me what you think. If anyone can also test with any type of mounted device and any transfer mode (always parallel, parallel for local files, parallel for local files on same devices only, never parallel), it could help to possibly find an unwanted behavior in a corner case.

Alexander Schwinn @alexxcons said:

Thanks alot for the changes, very nice !

I like the new enum, think it is more user-friendly now. You can add a tooltip in order to explain the details (e.g. that the copy will be started and directly paused)

I tried different copy/setting combinations, think I found one case which needs improvement: Started two copies from my SSD to a flashdrive, while having "only for local files" selected. They got copied in parallel. The file uris, if that may help to reproduce: Source (local ssd): /home/schwinn/software/thunar_test/Test1 /home/schwinn/software/thunar_test/Test2 Destination (USB flashdrive): /media/schwinn/USB_STICK/Test1 /media/schwinn/USB_STICK/Test2

Re-tested on some other flashdrive, same result: /media/schwinn/00BCB5F7BCB5E6F6/

On ething to mention: The other way around (copy from flashdrive to ssd) work well ! The second transfer is frozen.

... just as a side-node to ease testing. Creation of testfiles: fallocate -l 1G bigfile1

Cyrille Pontvieux @jrd said:

Thanks a lot for the testing.

I found the reason for the corner case and it should now be fixed in the latest version of the branch.

Thanks for fallocate tip, I was doing this with dd but it was a bit tedious.

Works fine now, thanks ! I'll try to review the mr in the next days

changed title from Freeze new copy where source or target device is shared with another running one to Support queued copy (configurable e.g. for remote devices)

closed via commit ead4f16e

Thanks alot for your contribution, @jrd , much appreciated !

Only the bugbounty part might not works any more now, since gitlab is not supported by bountysource. If you like, try to contact the donor directly !

mentioned in issue #116 (closed)

Ok I will. It was not my main concern, but it would be nice.

That was a long run, but I learned a lot of things along the way !! I would perhaps propose (on the ML, or IRC) some actions to help further contributions, especially first contributions by using my own feedback.

mentioned in issue #444 (closed)

Support queued copy (configurable e.g. for remote devices)

Submitted by Cyrille Pontvieux `@jrd`

Description

Depends on

See also

Designs

Child items 0

Activity

Support queued copy (configurable e.g. for remote devices)

Submitted by Cyrille Pontvieux @jrd

Description

Depends on

See also

Activity

Submitted by Cyrille Pontvieux `@jrd`