Sometimes you cannot change the current device in the pulseaudio server control plugin
Changing the device works through pactl, but if you do the same through UI, the device will change for a moment and then go back. The problem is solved only by closing a specific application that uses the device (captures the sound in my case).