Non-UTF8 encodings mess up some labels and texts

Although I picked the xfce4-panel as the culprit, the problem affects other components too: when the LANG is set to, for example, uk_UA.KOI8-U various labels and other textual entries get messed up badly.

Many, but not all. For example, the title of main application menu (called "Applications" in English), shows junk characters instead of Ukrainian "Програми" -- bad. However, in the popup menu itself, the entries are still listed in proper Cyrillic.

This seems like a 4.18 regression -- I still have 4.16 installed on a different machine, and everything looks good there. Changing LANG to uk_UA.UTF-8 fixes the problem, but I'd rather stick to the 8-bit encoding...

It feels related to the earlier bug 12294, but on a much greater scale

I can't reproduce this problem with xfce4-panel 4.18.x on Arch Linux. "Applications" is correctly displayed in Ukrainian as above in the tooltip and preferences dialog of the appmenu plugin.

Which panel version exactly are you using (xfce4-panel -V) and what is your distribution?

added 3. Need Info label

added 5. Applications Menu label

Which panel version exactly are you using (xfce4-panel -V)

Even that command produces garbled text, when LANG is set to uk_UA.KOI8-U:

% xfce4-panel -V
xfce4-panel 4.18.4 (Xfce 4.18)

Copyright (c) 2004-2023
[Invalid UTF-8]         \xeb\xcf\xcd\xc1\xce\xc4\xc1 \xd2\xcf\xda\xd2\xcf\xc2\xcb\xc9 Xfce. \xf7\xd3\xa6 \xd0\xd2\xc1\xd7\xc1 \xda\xc1\xc8\xc9\xdd\xc5\xce\xa6.

[Invalid UTF-8] \xf3\xd0\xcf\xd7\xa6\xd3\xd4\xa6\xd4\xd8 \xd0\xd2\xcf \xd0\xcf\xcd\xc9\xcc\xcb\xd5 \xce\xc1 <https://gitlab.xfce.org/xfce/xfce4-panel/-/issues>.

and what is your distribution?

Sorry, I don't understand this question. What's "distribution"?

On another machine, where I still have Xfce-4.16 installed, the version-text is output correctly:

% xfce4-panel -V
xfce4-panel 4.16.4 (Xfce 4.16)

Copyright (c) 2004-2022
        Команда розробки Xfce. Всі права захищені.

Сповістіть про помилку на <http://bugzilla.xfce.org/>.

Sorry, I don't understand this question. What's "distribution"?

Linux distribution :) https://en.wikipedia.org/wiki/Linux_distribution

Linux distribution

Oh... No, I'm not using Linux. The new setup, where I'm having a problem, runs FreeBSD-14/amd64. The older machine, where 4.16 works correctly, runs FreeBSD-13/amd64.

Hope, this helps.

Ah, I can reproduce the issue with xfce4-panel -V:

$ LANG=uk_UA.KOI8-U xfce4-panel -V
xfce4-panel 4.18.4 (Xfce 4.18)

Copyright (c) 2004-2023
	������� �������� Xfce. �Ӧ ����� ������Φ.

���צ�Ԧ�� ��� ������� �� <https://gitlab.xfce.org/xfce/xfce4-panel/-/issues>.

However it's not a regression for me:

$ LANG=uk_UA.KOI8-U xfce4-panel -V
xfce4-panel 4.16.5 (Xfce 4.18)

Copyright (c) 2004-2022
	������� �������� Xfce. �Ӧ ����� ������Φ.

���צ�Ԧ�� ��� ������� �� <http://bugzilla.xfce.org/>.

Or maybe the problem comes from libxfce4ui/util 4.18.x or something... Thanks for reporting anyway, I'll look into it.

removed 3. Need Info label

assigned to @Tamaranch

changed milestone to %4.18.5

removed 5. Applications Menu label

added 7. Localization label

Ah, I can reproduce the issue

It looks different for you -- there is no complaint about "Invalid UTF-8". Maybe, you just don't have Cyrillic fonts loaded into the terminal-emulator?

Or maybe the problem comes from libxfce4ui/util 4.18.x or something

It certainly is not just panel, yes...

I'll look into it

Thank you!

Maybe, you just don't have Cyrillic fonts loaded into the terminal-emulator?

No, it works in UTF-8:

$ LANG=uk_UA.UTF-8 xfce4-panel -V
xfce4-panel 4.18.4 (Xfce 4.18)

Copyright (c) 2004-2023
	Команда розробки Xfce. Всі права захищені.

Сповістіть про помилку на <https://gitlab.xfce.org/xfce/xfce4-panel/-/issues>.

I can't actually reproduce the problem. For the display in the terminal above to work correctly, I need to select the correct encoding in the options, which seems normal:

$ LANG=uk_UA.KOI8-U xfce4-panel -V
xfce4-panel 4.19.1git-a6c4e8a11 (Xfce 4.18)

Copyright (c) 2004-2023
	Команда розробки Xfce. Всі права захищені.

Сповістіть про помилку на <https://gitlab.xfce.org/xfce/xfce4-panel/-/issues>.

The only way I've found to reproduce what you're getting is to set the encoding to NULL in bind_textdomain_codeset() called from libxfce4util:

$ LANG=uk_UA.KOI8-U xfce4-panel -V
xfce4-panel 4.19.1git-a6c4e8a11 (Xfce 4.18)

Copyright (c) 2004-2023
[Invalid UTF-8] 	\xeb\xcf\xcd\xc1\xce\xc4\xc1 \xd2\xcf\xda\xd2\xcf\xc2\xcb\xc9 Xfce. \xf7\xd3\xa6 \xd0\xd2\xc1\xd7\xc1 \xda\xc1\xc8\xc9\xdd\xc5\xce\xa6.

[Invalid UTF-8] \xf3\xd0\xcf\xd7\xa6\xd3\xd4\xa6\xd4\xd8 \xd0\xd2\xcf \xd0\xcf\xcd\xc9\xcc\xcb\xd5 \xce\xc1 <https://gitlab.xfce.org/xfce/xfce4-panel/-/issues>.

But this code hasn't changed since 4.16, and even before. Anyway it looks like this function is the problem for you, but I don't know why. Perhaps another update, not directly related to Xfce, has changed this behavior between your two machines?

Is even this function still available on your system, i.e. do you pass this check?

removed milestone %4.18.5

unassigned @Tamaranch

added 3. Need Info label

Is even this function still available on your system, i.e. do you pass this check?

The setlocale() is found alright, but the bind_textdomain_codeset() is not! Here is the relevant section from config.log:

...
configure:14702: checking for bind_textdomain_codeset
configure:14702: cc -o conftest -O2 -pipe -march=sandybridge  -fstack-protector-strong -fno-strict-aliasing  -I/opt/include  -fstack-protector-strong  conftest.c -L/opt/lib >&5
ld: error: undefined symbol: bind_textdomain_codeset
>>> referenced by conftest.c
>>>               /tmp/conftest-a6dfc2.o:(main)
cc: error: linker command failed with exit code 1 (use -v to see invocation)
configure:14702: $? = 1
configure: failed program was:
...

The function is not found in libc (at least, on FreeBSD) -- it is provided by libintl (part of gettext), but XFce4's configure does not add -lintl to the compiler command-line, when testing for the function's presence

Is this the problem?

Is this the problem?

Yep, this was it... After forcing the use of -lintl here -- and rebuilding all of the installed xfce4-pieces, I can now view all the menus in Ukrainian again.

Please, correct the configure.ac for the next release. Thank you!

Not sure we have anything to fix here. I only see -I/opt/include and -L/opt/lib in the above compile command. Wouldn't it be enough to add /usr and/or /usr/local? See e.g. https://github.com/Snaipe/Criterion/issues/48#issuecomment-141505056.

I only see -I/opt/include and -L/opt/lib in the above compile command.

Yep -- the -lintl is missing, and that's why the symbol in question is not found by the linker.

The configure then assumes, bind_textdomain_codeset is missing on the platform and proceeds to implement its own fake one, which breaks things.

Hmm… still not sure we're the problem in this case. Moving to libxfce4util anyway.

closed

moved to libxfce4util#25 (closed)

added 3. Moved label

removed 3. Need Info label