keyboard settings periodically lost

I have the keyboard layouts configured via setxkbmap. The plugin displays flags and remembers the layout per application. The problem is that after some time when switching layouts the flag icon of the layout is empty and the layout is not remembered for the application. Running setxkbmap again fixes the problem, but after a while the settings are lost again.