Using the variable "$HOME" in the address bar will be expanded to the home directory.
https://bugzilla.xfce.org/show_bug.cgi?id=12165