Make desktop folder loading cancellable

This avoids an erroneous memory access caused by freeing icon-view-manager while callbacks are still in flight.

Fixes: #297 (closed).