The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Thanks to Mike Fabian for the great work!
Bug 1238406 - Glibc locale subpackaging https://bugzilla.redhat.com/show_bug.cgi?id=1238406
Changes/Glibc locale subpackaging https://fedoraproject.org//wiki/Changes/Glibc_locale_subpackaging
Notes: - If you don't use langpacks you won't get any locales except C, POSIX and C.UTF-8. - If you don't want to use langpacks just install the glibc langpack you want manually. e.g. dnf install glibc-langpack-en - Anaconda should take care of everything for you via langpacks. - We missed the fact that for the F23->F24 transition `dnf system-upgrade` should install glibc-all-langpacks (meta package) to give F23 users a smooth transition with all the locales they had before, but once in F24 the langpacks will be used. Filed: Bug 1312103 - “dnf system-upgrade” needs to install “glibc-all-langpacks” when upgrading from f23 to f24 https://bugzilla.redhat.com/show_bug.cgi?id=1312103
Cheers, Carlos.
You also forget to mention what Rawhide users are supposed to do.
I assume that I should do:
# dnf install langpacks-cs
to have only czech language available and English should be available by default, right?
Vít
Dne 26.2.2016 v 10:47 Carlos O'Donell napsal(a):
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Thanks to Mike Fabian for the great work!
Bug 1238406 - Glibc locale subpackaging https://bugzilla.redhat.com/show_bug.cgi?id=1238406
Changes/Glibc locale subpackaging https://fedoraproject.org//wiki/Changes/Glibc_locale_subpackaging
Notes:
- If you don't use langpacks you won't get any locales except C, POSIX and C.UTF-8.
- If you don't want to use langpacks just install the glibc langpack you want manually. e.g. dnf install glibc-langpack-en
- Anaconda should take care of everything for you via langpacks.
- We missed the fact that for the F23->F24 transition `dnf system-upgrade` should install glibc-all-langpacks (meta package) to give F23 users a smooth transition with all the locales they had before, but once in F24 the langpacks will be used. Filed: Bug 1312103 - “dnf system-upgrade” needs to install “glibc-all-langpacks” when upgrading from f23 to f24 https://bugzilla.redhat.com/show_bug.cgi?id=1312103
Cheers, Carlos. -- devel mailing list devel@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
On 02/26/2016 05:02 AM, Vít Ondruch wrote:
You also forget to mention what Rawhide users are supposed to do.
I assume that I should do:
# dnf install langpacks-cs
to have only czech language available and English should be available by default, right?
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove. So for F24 you get nothing by default, and dnf langpacks plugin does everything for you in the background.
If you're on rawhide, and you don't use langpacks, you can install the appropriate langpack by hand with `dnf install glibc-langpack-cs`.
Cheers, Carlos
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove.
For this reasons we have weak dependencies. I guess that: Recommends: glibc-all-langpacks can do the work (install or by default, can be safely removed). Or Suggests: glibc-all-langpacks so it is not installed by default, but users can get the information that it can enhance the usability of glibc.
On 02/26/2016 08:39 AM, Miroslav Suchý wrote:
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove.
For this reasons we have weak dependencies. I guess that: Recommends: glibc-all-langpacks can do the work (install or by default, can be safely removed). Or Suggests: glibc-all-langpacks so it is not installed by default, but users can get the information that it can enhance the usability of glibc.
Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks. The net result would be that unless a specific langpack was chosen, you'd end up with all of them to satisfy the requirement. (This would also unbreak the upgrades without needing a patch to dnf system-upgrade)
On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote:
On 02/26/2016 08:39 AM, Miroslav Suchý wrote:
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove.
For this reasons we have weak dependencies. I guess that: Recommends: glibc-all-langpacks can do the work (install or by default, can be safely removed). Or Suggests: glibc-all-langpacks so it is not installed by default, but users can get the information that it can enhance the usability of glibc.
Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks. The net result would be that unless a specific langpack was chosen, you'd end up with all of them to satisfy the requirement. (This would also unbreak the upgrades without needing a patch to dnf system-upgrade)
That would be a much better solution. dnf-system-upgrade so far didn't have any special handling for specific packages (except for a check that the kernel is in the upgrade transaction) and there no mechanism like this is implemented.
Zbyszek
On 02/26/2016 10:39 AM, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote:
On 02/26/2016 08:39 AM, Miroslav Suchý wrote:
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove.
For this reasons we have weak dependencies. I guess that: Recommends: glibc-all-langpacks can do the work (install or by default, can be safely removed). Or Suggests: glibc-all-langpacks so it is not installed by default, but users can get the information that it can enhance the usability of glibc.
Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks. The net result would be that unless a specific langpack was chosen, you'd end up with all of them to satisfy the requirement. (This would also unbreak the upgrades without needing a patch to dnf system-upgrade)
That would be a much better solution. dnf-system-upgrade so far didn't have any special handling for specific packages (except for a check that the kernel is in the upgrade transaction) and there no mechanism like this is implemented.
We are investigating switching to this. It shouldn't be a problem, and with a quick rebuild of glibc in F24 I think we'll be ready. I just need to run through the testing quickly on Monday.
All of our langpacks are auto-generated based on glibc supported locales so there is no grunt work in changing, just testing again that it works smoothly.
Cheers, Carlos.
On Sat, 2016-02-27 at 09:03 -0500, Carlos O'Donell wrote:
On 02/26/2016 10:39 AM, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote:
On 02/26/2016 08:39 AM, Miroslav Suchý wrote:
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove.
For this reasons we have weak dependencies. I guess that: Recommends: glibc-all-langpacks can do the work (install or by default, can be safely removed). Or Suggests: glibc-all-langpacks so it is not installed by default, but users can get the information that it can enhance the usability of glibc.
Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks. The net result would be that unless a specific langpack was chosen, you'd end up with all of them to satisfy the requirement. (This would also unbreak the upgrades without needing a patch to dnf system-upgrade)
That would be a much better solution. dnf-system-upgrade so far didn't have any special handling for specific packages (except for a check that the kernel is in the upgrade transaction) and there no mechanism like this is implemented.
We are investigating switching to this. It shouldn't be a problem, and with a quick rebuild of glibc in F24 I think we'll be ready. I just need to run through the testing quickly on Monday.
All of our langpacks are auto-generated based on glibc supported locales so there is no grunt work in changing, just testing again that it works smoothly.
I have filed https://bugzilla.redhat.com/show_bug.cgi?id=1312956 and nominated it as a Beta blocker, for the issue of all locales disappearing on upgrade.
On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote:
Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks.
This would force the installation of at least one langpack, no? The C, POSIX and *C.UTF-8* locales are builtin, so for many systems it is very reasonable to run without any language pack installed.
On Feb 28, 2016, at 5:08 PM, Lars Seipel lars.seipel@gmail.com wrote:
On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote: Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks.
This would force the installation of at least one langpack, no? The C, POSIX and *C.UTF-8* locales are builtin, so for many systems it is very reasonable to run without any language pack installed.
Yeah, the workaround we're doing there is to have a glibc-minimal-langpacks subpackage that satisfies the requirement but contains no actual files. It's a bit of a hack, but not a particularly awful one.
On Sunday, 28 February 2016 at 23:31, Stephen Gallagher wrote:
On Feb 28, 2016, at 5:08 PM, Lars Seipel lars.seipel@gmail.com wrote:
On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote: Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks.
This would force the installation of at least one langpack, no? The C, POSIX and *C.UTF-8* locales are builtin, so for many systems it is very reasonable to run without any language pack installed.
Yeah, the workaround we're doing there is to have a glibc-minimal-langpacks subpackage that satisfies the requirement but contains no actual files. It's a bit of a hack, but not a particularly awful one.
Why a subpackage? Shouldn't the main glibc package (or glibc-common) simply provide the C, C.UTF-8 and POSIX langpacks?
Regards, Dominik
On 03/05/2016 07:13 PM, Dominik 'Rathann' Mierzejewski wrote:
On Sunday, 28 February 2016 at 23:31, Stephen Gallagher wrote:
On Feb 28, 2016, at 5:08 PM, Lars Seipel lars.seipel@gmail.com wrote:
On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote: Yeah, I think the best approach would be to have all the langpacks offer a virtual Provides: glibc-langpack and have the main package Requires: glibc-langpack and Suggests: glibc-all-langpacks.
This would force the installation of at least one langpack, no? The C, POSIX and *C.UTF-8* locales are builtin, so for many systems it is very reasonable to run without any language pack installed.
Yeah, the workaround we're doing there is to have a glibc-minimal-langpacks subpackage that satisfies the requirement but contains no actual files. It's a bit of a hack, but not a particularly awful one.
Why a subpackage? Shouldn't the main glibc package (or glibc-common) simply provide the C, C.UTF-8 and POSIX langpacks?
The glibc-common package *does* provide the C, C.UTF-8 and POSIX langpacks. The reason for the subpackage is just a hack to fix some issues with upgrades.
We needed a way to ensure that upgrades would pull in the complete set of langpacks, since we had no way of knowing which ones were actually being used. The way we did this was to add a "Requires: glibc-langpack" to the glibc package and a "Suggests: glibc-all-langpacks" as well. Then we added "Provides: glibc-langpack" to all of the subpackages that provide a language. However, this resulted in not being able to restrict it to JUST the built-ins, so we added a special, empty glibc-minimal-langpacks subpackage whose purpose is just to satisfy the "Requires: glib-langpack" dependency. It contains no content and doesn't add anything to the hard drive except the RPM database.
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
On 02/26/2016 05:02 AM, Vít Ondruch wrote:
You also forget to mention what Rawhide users are supposed to do.
I assume that I should do:
# dnf install langpacks-cs
to have only czech language available and English should be available by default, right?
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove. So for F24 you get nothing by default, and dnf langpacks plugin does everything for you in the background.
If you're on rawhide, and you don't use langpacks, you can install the appropriate langpack by hand with `dnf install glibc-langpack-cs`.
This is very badly communicated I must say. I really don't understand, why the suggested steps are not:
1) dnf install langpack-[your-language] 2) dnf update glibc*
This should be the workflow and not "you use" or "you don't use". This is not something which will be glibc specific. The langpack-[your-language] is precondition to make this work, not to suggest installing glibc-langapack-[your-language] or even glibc-all-langpacks.
I am bit annoyed by this, because doing just "dnf update" will break the system in a way that gnome-terminal won't start anymore, DNF is totally busted etc. so simple instructions would be really appreciated.
Vít
On 02/29/2016 04:22 AM, Vít Ondruch wrote:
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
On 02/26/2016 05:02 AM, Vít Ondruch wrote:
You also forget to mention what Rawhide users are supposed to do.
I assume that I should do:
# dnf install langpacks-cs
to have only czech language available and English should be available by default, right?
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove. So for F24 you get nothing by default, and dnf langpacks plugin does everything for you in the background.
If you're on rawhide, and you don't use langpacks, you can install the appropriate langpack by hand with `dnf install glibc-langpack-cs`.
This is very badly communicated I must say. I really don't understand, why the suggested steps are not:
- dnf install langpack-[your-language]
- dnf update glibc*
This should be the workflow and not "you use" or "you don't use". This is not something which will be glibc specific. The langpack-[your-language] is precondition to make this work, not to suggest installing glibc-langapack-[your-language] or even glibc-all-langpacks.
I am bit annoyed by this, because doing just "dnf update" will break the system in a way that gnome-terminal won't start anymore, DNF is totally busted etc. so simple instructions would be really appreciated.
I agree, and I apologize. We're working on an rpm dependency based resolution process which will fix this shortly.
Cheers, Carlos.
Dne 29.2.2016 v 17:12 Carlos O'Donell napsal(a):
On 02/29/2016 04:22 AM, Vít Ondruch wrote:
Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
On 02/26/2016 05:02 AM, Vít Ondruch wrote:
You also forget to mention what Rawhide users are supposed to do.
I assume that I should do:
# dnf install langpacks-cs
to have only czech language available and English should be available by default, right?
No languages are available by default, we did this because otherwise you keep carrying forward locales that you can't remove. So for F24 you get nothing by default, and dnf langpacks plugin does everything for you in the background.
If you're on rawhide, and you don't use langpacks, you can install the appropriate langpack by hand with `dnf install glibc-langpack-cs`.
This is very badly communicated I must say. I really don't understand, why the suggested steps are not:
- dnf install langpack-[your-language]
- dnf update glibc*
This should be the workflow and not "you use" or "you don't use". This is not something which will be glibc specific. The langpack-[your-language] is precondition to make this work, not to suggest installing glibc-langapack-[your-language] or even glibc-all-langpacks.
I am bit annoyed by this, because doing just "dnf update" will break the system in a way that gnome-terminal won't start anymore, DNF is totally busted etc. so simple instructions would be really appreciated.
I agree, and I apologize. We're working on an rpm dependency based resolution process which will fix this shortly.
Although I was grumpy, I would also like to thank you for working on this, since this is definitely step in the right direction. I appreciate your effort. Thanks.
Vít
On 02/26/2016 04:47 AM, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Unfortunately we missed a piece of the puzzle here.
Bug 1303034 - rpm macro expansion works incorrectly when looping over a long list using lua https://bugzilla.redhat.com/show_bug.cgi?id=1303034
We need this fixed in F24 and pushed into the buildroots to unblock building glibc.
Cheers, Carlos.
Hi,
On Fri, Feb 26, 2016 at 3:52 PM, Carlos O'Donell carlos@redhat.com wrote:
On 02/26/2016 04:47 AM, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Unfortunately we missed a piece of the puzzle here.
Bug 1303034 - rpm macro expansion works incorrectly when looping over a long list using lua https://bugzilla.redhat.com/show_bug.cgi?id=1303034
We need this fixed in F24 and pushed into the buildroots to unblock building glibc.
I thought this is already fixed in F24. At least I remember I tested langpacks glibc.spec from mfabian's some old private git repository on F24 machine. The rpm build I tested was rpm-4.13.0-0.rc1.23.fc24. But now I see that f24 glibc is failing while comparing the SUPPORTED file in %prep.
Regards, Parag.
On 02/26/2016 07:33 AM, Parag Nemade wrote:
Hi,
On Fri, Feb 26, 2016 at 3:52 PM, Carlos O'Donell carlos@redhat.com wrote:
On 02/26/2016 04:47 AM, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Unfortunately we missed a piece of the puzzle here.
Bug 1303034 - rpm macro expansion works incorrectly when looping over a long list using lua https://bugzilla.redhat.com/show_bug.cgi?id=1303034
We need this fixed in F24 and pushed into the buildroots to unblock building glibc.
I thought this is already fixed in F24. At least I remember I tested langpacks glibc.spec from mfabian's some old private git repository on F24 machine. The rpm build I tested was rpm-4.13.0-0.rc1.23.fc24. But now I see that f24 glibc is failing while comparing the SUPPORTED file in %prep.
We didn't get that far in last nights build. I'll go look immediately.
Cheers, Carlos.
On 26 February 2016 at 09:47, Carlos O'Donell carlos@redhat.com wrote:
We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf.
What happens if the user isn't using DNF? For the workstation we have to support users using just the graphical tools, and we can't rely on command line tools for this kind of thing. I'm happy to add support for installing lang packs into gnome-software but need to know how the langpack plugin works so we can add proper support into the AppStream metadata file. The idea being if the user changes the per-session language we install the required langpacks automatically.
Can someone explain how the dnf langpack plugin gets the data on what application subpackages to install? It would be easy if we could have a virtual package that we could install for each locale as then we can just put the metainfo file there for easy support in gnome-software.
Thanks,
Richard
What happens if the user isn't using DNF? For the workstation we have to support users using just the graphical tools, and we can't rely on command line tools for this kind of thing. I'm happy to add support for installing lang packs into gnome-software but need to know how the langpack plugin works so we can add proper support into the AppStream metadata file. The idea being if the user changes the per-session language we install the required langpacks automatically.
Can someone explain how the dnf langpack plugin gets the data on what application subpackages to install? It would be easy if we could have a virtual package that we could install for each locale as then we can just put the metainfo file there for easy support in gnome-software.
My understanding this is implementing using the same langpack functionality that has been there for ages even in yum for all sorts of lang support, such as dictionaries, grammar etc, it's just been that glibc has never been able to split out the language support in the past.
Peter
Hi Richard,
On Fri, Feb 26, 2016 at 5:55 PM, Richard Hughes hughsient@gmail.com wrote:
On 26 February 2016 at 09:47, Carlos O'Donell carlos@redhat.com wrote:
We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf.
What happens if the user isn't using DNF? For the workstation we have to support users using just the graphical tools, and we can't rely on command line tools for this kind of thing. I'm happy to add support for installing lang packs into gnome-software but need to know how the langpack plugin works so we can add proper support into the AppStream metadata file. The idea being if the user changes the per-session language we install the required langpacks automatically.
Can someone explain how the dnf langpack plugin gets the data on what application subpackages to install? It would be easy if we could have a virtual package that we could install for each locale as then we can just put the metainfo file there for easy support in gnome-software.
Please have a look at New Langpacks Change[1] information. With this change we are trying to remove the need of having dnf langpacks plugin. The basic required things are already in F24+ but I have not yet got time to add weak dependencies in all the required langpacks. But I have modified basic required langpacks like hunspell, hyphen, mythes. Then libreoffice, kde-l10n, gnome-getting-started-docs and tesseract has been also built for this new langpacks change. I do have created test page[2] for this new change.
I suppose all the langpacks changes will be complete by next week. Please note that I specifically requested Workstation WG to add dnf-langpacks package in comps but later on we decided to use weak dependencies for langpacks. Now users need to just install meta-packages like langpacks-<langcode> to get all the required langpacks installed on the system. If needed we can remove dnf-langpacks from Fedora Workstation installation but I will say better wait till Fedora 24 Alpha release and see if new langpacks implementation has worked fine by that time.
Regards, Parag
[1] https://fedoraproject.org/wiki/Changes/LangpacksInstallationWithRPMWeakDepen... [2] https://fedoraproject.org/wiki/QA:Langpacks
On 26 February 2016 at 13:25, Parag Nemade panemade@gmail.com wrote:
Please have a look at New Langpacks Change[1] information.
Thanks for the information -- this looks a lot better than the current logic and so a huge thanks for driving this. One more question: Would you be against me adding (translated) metainfo files to each of the langpack subpackages? This would allow me to add support in GNOME Software very easily without any hacky code. I'm happy doing the work if this sounds acceptable to you.
Richard
Hi,
On Fri, Feb 26, 2016 at 8:54 PM, Richard Hughes hughsient@gmail.com wrote:
On 26 February 2016 at 13:25, Parag Nemade panemade@gmail.com wrote:
Please have a look at New Langpacks Change[1] information.
Thanks for the information -- this looks a lot better than the current logic and so a huge thanks for driving this. One more question: Would you be against me adding (translated) metainfo files to each of the langpack subpackages? This would allow me to add support in GNOME Software very easily without any hacky code. I'm happy doing the work if this sounds acceptable to you.
I am fine with you adding metainfo files to langpacks.spec file.
Regards, Parag.
On Fri, 2016-02-26 at 12:25 +0000, Richard Hughes wrote:
What happens if the user isn't using DNF? For the workstation we have to support users using just the graphical tools, and we can't rely on command line tools for this kind of thing. I'm happy to add support for installing lang packs into gnome-software but need to know how the langpack plugin works so we can add proper support into the AppStream metadata file. The idea being if the user changes the per-session language we install the required langpacks automatically.
Can someone explain how the dnf langpack plugin gets the data on what application subpackages to install? It would be easy if we could have a virtual package that we could install for each locale as then we can just put the metainfo file there for easy support in gnome-software.
We also need locale selection in gnome-control-center and gnome- initial-setup to work. Any chance you'll work on that too, Richard?
Otherwise, we should be able to resolve this by adding glibc-all- langpacks to the Workstation group in comps (correct Carlos?). It defeats the point of this change, but releasing with broken locale selection is not gonna happen.
Michael
On 02/26/2016 09:44 AM, Michael Catanzaro wrote:
On Fri, 2016-02-26 at 12:25 +0000, Richard Hughes wrote:
What happens if the user isn't using DNF? For the workstation we have to support users using just the graphical tools, and we can't rely on command line tools for this kind of thing. I'm happy to add support for installing lang packs into gnome-software but need to know how the langpack plugin works so we can add proper support into the AppStream metadata file. The idea being if the user changes the per-session language we install the required langpacks automatically.
Can someone explain how the dnf langpack plugin gets the data on what application subpackages to install? It would be easy if we could have a virtual package that we could install for each locale as then we can just put the metainfo file there for easy support in gnome-software.
We also need locale selection in gnome-control-center and gnome- initial-setup to work. Any chance you'll work on that too, Richard?
Otherwise, we should be able to resolve this by adding glibc-all- langpacks to the Workstation group in comps (correct Carlos?). It defeats the point of this change, but releasing with broken locale selection is not gonna happen.
Correct, 'glibc-all-langpacks' will solve the problem immediately.
We're working right now on the solution Stephen suggested which is a weak ref/suggests which looks like it would let rpm handle the upgrade properly.
Cheers, Carlos.
On Fri, 2016-02-26 at 10:06 -0500, Carlos O'Donell wrote:
Correct, 'glibc-all-langpacks' will solve the problem immediately.
OK, I've added this to comps, to take care of new installs. If anyone implements support for language pack installation in both gnome- initial-setup and gnome-control-center, we can remove this.
We're working right now on the solution Stephen suggested which is a weak ref/suggests which looks like it would let rpm handle the upgrade properly.
And I trust you guys will take care of upgrades. Thanks!
Michael
On 02/26/2016 10:38 AM, Michael Catanzaro wrote:
On Fri, 2016-02-26 at 10:06 -0500, Carlos O'Donell wrote:
Correct, 'glibc-all-langpacks' will solve the problem immediately.
OK, I've added this to comps, to take care of new installs. If anyone implements support for language pack installation in both gnome- initial-setup and gnome-control-center, we can remove this.
We're working right now on the solution Stephen suggested which is a weak ref/suggests which looks like it would let rpm handle the upgrade properly.
And I trust you guys will take care of upgrades. Thanks!
We've also had some questions about the performance and disk space of the split up langpacks. So I'm going to work on a blog post to help clarify this for everyone.
Cheers, Carlos.
On Fri, Feb 26, 2016 at 10:47 AM, Carlos O'Donell carlos@redhat.com wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Thanks to Mike Fabian for the great work!
I'm now seeing this in mock / koji builds:
perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LANG = "en_US.UTF-8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C").
For example https://kojipkgs.fedoraproject.org//packages/collectd/5.5.1/4.fc25/data/logs...
It looks harmless, but could this be related to this change and is this already a known issue?
Kind regards,
Ruben Kerkhof
On 2016-02-27, Ruben Kerkhof ruben@rubenkerkhof.com wrote:
I'm now seeing this in mock / koji builds:
perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LANG = "en_US.UTF-8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C").
Mock (or rpmbuild?) sets LANG to en_US.UTF-8 instead of C in order to have UTF-8 locale when building packages because it was decided that UTF-8 is the default (build) environment for Fedora.
With removing en_US.UTF-8 from the glibc and thus minimal build root, all koji builds are misconfigured now because setlocale(3) on "en_US.UTF-8" fails without the locale definition in the file system.
I assume changing the default build-time locale to C.UTF-8 could help.
-- Petr
On 02/29/2016 06:32 AM, Petr Pisar wrote:
On 2016-02-27, Ruben Kerkhof ruben@rubenkerkhof.com wrote:
I'm now seeing this in mock / koji builds:
perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LANG = "en_US.UTF-8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C").
Mock (or rpmbuild?) sets LANG to en_US.UTF-8 instead of C in order to have UTF-8 locale when building packages because it was decided that UTF-8 is the default (build) environment for Fedora.
With removing en_US.UTF-8 from the glibc and thus minimal build root, all koji builds are misconfigured now because setlocale(3) on "en_US.UTF-8" fails without the locale definition in the file system.
I assume changing the default build-time locale to C.UTF-8 could help.
The glibc team guarantees you will have C.UTF-8 always.
It should be the default.
This is indeed related to the langpack split up and we are going to fix this shortly.
Cheers, Carlos.
Excerpts from Carlos O'Donell's message of 2016-02-29 11:04 -05:00:
On 02/29/2016 06:32 AM, Petr Pisar wrote:
Mock (or rpmbuild?) sets LANG to en_US.UTF-8 instead of C in order to have UTF-8 locale when building packages because it was decided that UTF-8 is the default (build) environment for Fedora.
With removing en_US.UTF-8 from the glibc and thus minimal build root, all koji builds are misconfigured now because setlocale(3) on "en_US.UTF-8" fails without the locale definition in the file system.
One of my Python packages' tests failed in Koschei due to (I think) the same issue. The tests were failing in Python 3 because now sys.stdout is encoded as ASCII whereas it (presumably) should be UTF-8.
I assume changing the default build-time locale to C.UTF-8 could help.
The glibc team guarantees you will have C.UTF-8 always.
It should be the default.
This is indeed related to the langpack split up and we are going to fix this shortly.
When you say it will be fixed, do you mean fixed by changing Koji to run all builds in C.UTF-8 instead of en_US.UTF-8? Or fixed by ensuring that en_US.UTF-8 is available in the build environment (by adding it to the Koji build group I guess)? Or some other fix?
On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Thanks to Mike Fabian for the great work!
This did completely break anaconda (and thus also broke live image and Cloud image creation).
https://bugzilla.redhat.com/show_bug.cgi?id=1312607
And also I was not able to start gnome-terminal and some other apps. I had to install glibc-langpack-en.
On Sat, Feb 27, 2016, 9:44 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Thanks to Mike Fabian for the great work!
This did completely break anaconda (and thus also broke live image and Cloud image creation).
https://bugzilla.redhat.com/show_bug.cgi?id=1312607
Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net
-- devel mailing list devel@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
On Sat, 2016-02-27 at 20:57 +0000, Igor Gnatenko wrote:
And also I was not able to start gnome-terminal and some other apps. I had to install glibc-langpack-en.
Different case. The issue of how to handle upgrade of existing installs is already being discussed upthread. I was pointing out a different problem.
On Sat, Feb 27, 2016 at 08:57:27PM +0000, Igor Gnatenko wrote:
And also I was not able to start gnome-terminal and some other apps. I had to install glibc-langpack-en.
libvirtd also fails to start up:
https://bugzilla.redhat.com/show_bug.cgi?id=1312688
I'm investigating it now.
Rich.
On Mon, Feb 29, 2016 at 01:42:57PM +0000, Richard W.M. Jones wrote:
On Sat, Feb 27, 2016 at 08:57:27PM +0000, Igor Gnatenko wrote:
And also I was not able to start gnome-terminal and some other apps. I had to install glibc-langpack-en.
libvirtd also fails to start up:
https://bugzilla.redhat.com/show_bug.cgi?id=1312688
I'm investigating it now.
Is it OK to ignore the return value of setlocale()?
Rich.
On 02/29/2016 10:46 AM, Richard W.M. Jones wrote:
On Mon, Feb 29, 2016 at 01:42:57PM +0000, Richard W.M. Jones wrote:
On Sat, Feb 27, 2016 at 08:57:27PM +0000, Igor Gnatenko wrote:
And also I was not able to start gnome-terminal and some other apps. I had to install glibc-langpack-en.
libvirtd also fails to start up:
https://bugzilla.redhat.com/show_bug.cgi?id=1312688
I'm investigating it now.
Is it OK to ignore the return value of setlocale()?
Yes and no.
It's a quality issue.
Without the locale your localizations like dates may be in formats the user doesn't expect.
It's up to you to decide what quality you consider "critical."
We are going to fix this shortly in F24+ though so that more locales are installed during the upgrade.
Cheers, Carlos.
On 02/27/2016 03:44 PM, Adam Williamson wrote:
On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Thanks to Mike Fabian for the great work!
This did completely break anaconda (and thus also broke live image and Cloud image creation).
The glibc team is working to put out an ASAP update that would use a slightly tweaked set of rpm dependencies to ensure glibc-all-lanpacks is pulled in by default in more cases.
We appreciate your patience.
Cheers, Carlos.
On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Also in unpredictable fallout: on upgraded systems, ssh stops working until you install a langpack. At least it did for me.
On 02/27/2016 08:33 PM, Adam Williamson wrote:
On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Also in unpredictable fallout: on upgraded systems, ssh stops working until you install a langpack. At least it did for me.
That should never happen. What failed?
Cheers, Carlos.
On Mon, 2016-02-29 at 11:10 -0500, Carlos O'Donell wrote:
On 02/27/2016 08:33 PM, Adam Williamson wrote:
On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf. This greatly reduces the size of a glibc install from 130MB down to a couple of megs. It drops support for the hacky %_inst_langs feature, and relies entirely on langpack support (it was one or the other).
Also in unpredictable fallout: on upgraded systems, ssh stops working until you install a langpack. At least it did for me.
That should never happen. What failed?
IIRC it would simply terminate claiming the remote host had dropped the connection. There wasn't any useful information even with -vvvv. I just kinda guessed it must have something to do with the langpacks, installed glibc-langpack-en, and it started working again immediately. I can't reproduce the problem with 'LC_ALL=C ssh (somewhere)', so I can't immediately dig into it any more.
On 02/29/2016 05:18 PM, Adam Williamson wrote:
IIRC it would simply terminate claiming the remote host had dropped the connection. There wasn't any useful information even with -vvvv. I just kinda guessed it must have something to do with the langpacks, installed glibc-langpack-en, and it started working again immediately. I can't reproduce the problem with 'LC_ALL=C ssh (somewhere)', so I can't immediately dig into it any more.
Could you try with “LC_ALL=de_DE.UTF-8 ssh …”? Assuming that de_DE.UTF-8 is not installed on the target.
Other distributions already lack a full complement of locales in default installations, and most software works reasonably well in the presence of unsupported locale settings. It's certainly something we need to iron out.
Florian
On 02/29/2016 01:18 PM, Florian Weimer wrote:
On 02/29/2016 05:18 PM, Adam Williamson wrote:
IIRC it would simply terminate claiming the remote host had dropped the connection. There wasn't any useful information even with -vvvv. I just kinda guessed it must have something to do with the langpacks, installed glibc-langpack-en, and it started working again immediately. I can't reproduce the problem with 'LC_ALL=C ssh (somewhere)', so I can't immediately dig into it any more.
Could you try with “LC_ALL=de_DE.UTF-8 ssh …”? Assuming that de_DE.UTF-8 is not installed on the target.
Other distributions already lack a full complement of locales in default installations, and most software works reasonably well in the presence of unsupported locale settings. It's certainly something we need to iron out.
Agreed. The similar gnome-terminal failure should be fixed e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1312690
Cheers, Carlos.
On Mon, 2016-02-29 at 19:18 +0100, Florian Weimer wrote:
On 02/29/2016 05:18 PM, Adam Williamson wrote:
IIRC it would simply terminate claiming the remote host had dropped the connection. There wasn't any useful information even with -vvvv. I just kinda guessed it must have something to do with the langpacks, installed glibc-langpack-en, and it started working again immediately. I can't reproduce the problem with 'LC_ALL=C ssh (somewhere)', so I can't immediately dig into it any more.
Could you try with “LC_ALL=de_DE.UTF-8 ssh …”? Assuming that de_DE.UTF-8 is not installed on the target.
Other distributions already lack a full complement of locales in default installations, and most software works reasonably well in the presence of unsupported locale settings. It's certainly something we need to iron out.
Yeah, that does the trick:
[adamw@adam comps (master %)]$ LC_ALL=de_DE.UTF-8 ssh www /bin/bash: warning: setlocale: LC_ALL: cannot change locale (de_DE.UTF-8) ssh_exchange_identification: Connection closed by remote host [adamw@adam comps (master %)]$ ssh www Last login: Mon Feb 29 08:16:59 2016 from 192.168.1.5
note this is on Rawhide, not F24. Dunno if it affects F24.
On Mon, 2016-02-29 at 11:28 -0800, Adam Williamson wrote:
On Mon, 2016-02-29 at 19:18 +0100, Florian Weimer wrote:
On 02/29/2016 05:18 PM, Adam Williamson wrote:
IIRC it would simply terminate claiming the remote host had dropped the connection. There wasn't any useful information even with -vvvv. I just kinda guessed it must have something to do with the langpacks, installed glibc-langpack-en, and it started working again immediately. I can't reproduce the problem with 'LC_ALL=C ssh (somewhere)', so I can't immediately dig into it any more.
Could you try with “LC_ALL=de_DE.UTF-8 ssh …”? Assuming that de_DE.UTF-8 is not installed on the target.
Other distributions already lack a full complement of locales in default installations, and most software works reasonably well in the presence of unsupported locale settings. It's certainly something we need to iron out.
Yeah, that does the trick:
[adamw@adam comps (master %)]$ LC_ALL=de_DE.UTF-8 ssh www /bin/bash: warning: setlocale: LC_ALL: cannot change locale (de_DE.UTF-8) ssh_exchange_identification: Connection closed by remote host [adamw@adam comps (master %)]$ ssh www Last login: Mon Feb 29 08:16:59 2016 from 192.168.1.5
note this is on Rawhide, not F24. Dunno if it affects F24.
https://bugzilla.redhat.com/show_bug.cgi?id=1313041
Upgrading to this version of glibc also breaks postgres if you have a database that uses something other than the C locale. Installing appropriate language packs and restarting fix the issue.
On 02/26/2016 04:47 AM, Carlos O'Donell wrote:
Bug 1238406 - Glibc locale subpackaging https://bugzilla.redhat.com/show_bug.cgi?id=1238406
Changes/Glibc locale subpackaging https://fedoraproject.org//wiki/Changes/Glibc_locale_subpackaging
With glibc-2.23.90-3.fc25 and glibc-2.23.1-5.fc24 we have now fixed the issue in several important ways.
The `glibc-all-langpacks` continues to be the correct way to get all locales, but now things are a bit smoother.
(1) Virtual provides to make upgrade smoother.
All glibc languages packs now provide `glibc-langpack`, that includes the glibc-all-langpacks (all language support).
The core runtime depends on `glibc-langpack` and so when upgrading on systems without dnf, yum, or langpack plugin support, plain rpm dependency solving via suggests will install `glibc-all-langpack` for you and provide all the existing locales you previously had.
This makes for a complete rpm solution with no other support required.
This means that upgrading from a non-langpack supporting F23 to any future langpack supporting distribution will upgrade you to the newest set of supported locales.
(2) Optimized "all languages" performance.
In addition the `glibc-all-langpack` is optimized to again use the locale cache (locale-archive) to accelerate lookups for those users who prefer to have all languages installed. Previously we had `glibc-all-langpack` as a meta-package of all the other language packs, but that is no longer the case.
(3) Provide glibc-minimal-langpack
We add a minimal language pack that provides no languages. This allows you to install glibc-minimal-langpack and then uninstall glibc-all-langpacks and operate entirely under C, POSIX, and C.UTF-8 locales.
This should resolve all problems that users have been seeing with the upgrade.
Thank you for your patience. We really messed up here. In the future we'll make sure we cover all the upgrade cases more carefully.
(4) Remove all languages to minimize installed size
You can still minimize your installed size, but it's slighltly harder now. First you must install a valid language pack, and then remove `glibc-all-langpacks` to reduce the size of installed languages.
Cheers, Carlos.
Le 26/02/2016 10:47, Carlos O'Donell a écrit :
The glibc in Fedora rawhide and F24 is now split by language packs. We have over 180 supported languages in glibc, and those have been split into langpacks for transparent install and support via dnf.
I may understand the need.
But what about "server" ?
If you install a webapp which use gettext, the locale will be set according to the conneected user (ex Horde)
Does that mean that such webapp should requires "qlibc-all-langpacks" ?
Remi.
On Wed, Mar 09, 2016 at 07:53:44AM +0100, Remi Collet wrote:
I may understand the need. But what about "server" ? If you install a webapp which use gettext, the locale will be set according to the conneected user (ex Horde) Does that mean that such webapp should requires "qlibc-all-langpacks" ?
I'm not sure of the overall answer, but definitely please don't make it a _hard_ Requires — use Suggests or Recommends instead.