Mouse cursor graphics is misaligned by 1 pixel vertically and 1 pixel horizontally
First off, I will admit that this is almost certainly some sort of upstream issue as it occurs not only on Xubuntu 22.04 but also on LMDE5 (Cinnamon) and Manjaro KDE 21.2.6-220416 (UPDATE: it also occurs on Ubuntu 22.04 and Fedora 35 with a Ryzen 4800U's iGPU)... but I have no idea where to even begin with reporting this issue so I figured I'd start here since I myself use XFCE.
Anyway, the actual issue is that the mouse cursor graphic itself seems to be misaligned by 1 pixel horizontally and 1 pixel vertically. In other words, where the OS itself thinks the mouse cursor is ends up being not the upper left-most point of the cursor aka pixel coordinates of x=1,y=1 but rather pixel coordinates of x=2,y=2.
This is particularly noticeable if you're running at a lower resolution (especially really low resolutions like 640x480 or 800x600) and you move the cursor all the way to the edge of the screen. If you put your cursor in the top-left corner, the top row and left row of pixels on the cursor gets completely cut off yet, if you put your cursor on the bottom-right corner, 2x2 pixels will still be visible. Similarly, if you move the cursor to the middle of the right edge of the screen, 2 vertical rows of pixels will be visible.
Also, when you put the mouse cursor in the very top-left corner or the very top-right corner and do a left click-and-hold to make a selection rectangle, you can see that the selection rectangle will in fact go to the very edges and corners of the screen, suggesting that it really is just the cursor graphic that is misaligned rather than the actual physical location of your mouse.
It's important to note however that, if your run a Windows guest inside VirtualBox (windowed or fullscreen) on an XFCE desktop, this cursor misalignment does not occur inside the Windows guest but immediately does occur as soon as you move your cursor out of the Windows guest VM and into VirtualBox's actual toolbars.
EDIT: I had trouble taking screenshots capturing this issue, so I had to "cheat" by re-creating simulations of what it looks like.