Scroll event deltas are lost under certain circumstances
The original issue is here: xfce4-panel#411 (moved)
I opened an upstream bug after isolating the problem, where there is a short description: https://gitlab.gnome.org/GNOME/gtk/-/issues/4866
But it turns out that it is not reproducible in a GNOME/X11 session, and I can't reproduce the problem with Openbox either. So it could be related to Xfwm4 after all.
Related X11 issue: https://gitlab.freedesktop.org/xorg/xserver/-/issues/566