One of the planned parts of the F21 System Wide Change: BerkeleyDB 6 [1] is the introduction of downstream symbol versioning of both versions of the libraries (libdb with v6 and libdb5 with v5). This part is planned in order to not introduce bugs similar to [2]. However, if we introduce downstream versioning (as upstream is generally unresponsive), then we face the problem similar to [3].
In short, if we introduce the downstream versioning, we will ship library with ABI incompatible with upstream ABI. If we won't, applications with modules/plugins (ie. Apache with mod_perl) that each use different version of the library may break due to double symbols (one from the v5 and other from v6, and ld would not know which symbol is which).
I'd like to ask for suggestions on how to resolve this problem.
The ideal solution would be to convince upstream to version their symbols (and I will contact them about it), but from my experience with them this is very unlikely. Or we could try to keep an eye on troublesome applications and force them and all their modules/plugins to be built with the same version of libdb, but I have no idea if this is even possible.
As I stated above, I welcome any suggestions. I would also like to hear from someone responsible for the distro architecture which of the aforementioned issues is more painful for us, so we know which path to take if no complete solution is found.
Best regards and thanks for replies, Jan
[1] https://fedoraproject.org/wiki/Changes/BerkeleyDB_6 [2] https://bugzilla.redhat.com/show_bug.cgi?id=768846 [3] https://bugzilla.redhat.com/show_bug.cgi?id=1045013
On 05/07/2014 03:19 PM, Jan Staněk wrote:
One of the planned parts of the F21 System Wide Change: BerkeleyDB 6 [1] is the introduction of downstream symbol versioning of both versions of the libraries (libdb with v6 and libdb5 with v5). This part is planned in order to not introduce bugs similar to [2]. However, if we introduce downstream versioning (as upstream is generally unresponsive), then we face the problem similar to [3].
If upstream is unresponsive, why even upgrade to a new version with licensing-related challenges?
Anyway, have you investigated if you can use the upstream symbol mangling mechanism?
https://lists.fedoraproject.org/pipermail/devel/2014-April/198161.html
On Wed, May 07, 2014 at 03:19:42PM +0200, Jan Staněk wrote:
One of the planned parts of the F21 System Wide Change: BerkeleyDB 6 [1] is the introduction of downstream symbol versioning of both versions of the libraries (libdb with v6 and libdb5 with v5). This part is planned in order to not introduce bugs similar to [2]. However, if we introduce downstream versioning (as upstream is generally unresponsive), then we face the problem similar to [3].
If we keep libdb5 (forever?) then the [3] problem is less of an issue, IMO. Given that portably linking against libdb has historically been a major headache I'd be surprised if there are any binary third-party apps which even try doing this, to be honest.
I don't claim to understand the licensing matrix here, though, is there any background reading on that? Can we even switch *anything* which is potentially linked into httpd to using BDB 6? If not then basically nothing in $libdir can link against BDB 6.
Regards, Joe
On 07/05/14 15:39, Joe Orton wrote:
On Wed, May 07, 2014 at 03:19:42PM +0200, Jan Staněk wrote:
One of the planned parts of the F21 System Wide Change: BerkeleyDB 6 [1] is the introduction of downstream symbol versioning of both versions of the libraries (libdb with v6 and libdb5 with v5). This part is planned in order to not introduce bugs similar to [2]. However, if we introduce downstream versioning (as upstream is generally unresponsive), then we face the problem similar to [3].
If we keep libdb5 (forever?) then the [3] problem is less of an issue, IMO. Given that portably linking against libdb has historically been a major headache I'd be surprised if there are any binary third-party apps which even try doing this, to be honest.
for a data point, see this OOo commit from 2007 which equips the (bundled, of course) libdb*.so with a custom version script just to avoid conflicts and resulting crashes:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=cd83aeb5fdde8184aa82...
Hello, 2014-05-07 15:19 GMT+02:00 Jan Staněk jstanek@redhat.com:
In short, if we introduce the downstream versioning, we will ship library with ABI incompatible with upstream ABI. If we won't, applications with modules/plugins (ie. Apache with mod_perl) that each use different version of the library may break due to double symbols (one from the v5 and other from v6, and ld would not know which symbol is which).
I'd like to ask for suggestions on how to resolve this problem.
<snip>
As I stated above, I welcome any suggestions. I would also like to hear from someone responsible for the distro architecture which of the aforementioned issues is more painful for us, so we know which path to take if no complete solution is found.
In my personal opinion, keeping Fedora working at all is more important than cross-distribution compatibility (which is desirable but not as critical *to Fedora)*, so the former would be preferable.
Separately, please note today's FESCo request in https://fedorahosted.org/fesco/ticket/1291#comment:4 . Mirek
On Wed, 2014-05-07 at 15:19 +0200, Jan Staněk wrote:
One of the planned parts of the F21 System Wide Change: BerkeleyDB 6 [1] is the introduction of downstream symbol versioning of both versions of the libraries (libdb with v6 and libdb5 with v5). This part is planned in order to not introduce bugs similar to [2]. However, if we introduce downstream versioning (as upstream is generally unresponsive), then we face the problem similar to [3].
In short, if we introduce the downstream versioning, we will ship library with ABI incompatible with upstream ABI. If we won't, applications with modules/plugins (ie. Apache with mod_perl) that each use different version of the library may break due to double symbols (one from the v5 and other from v6, and ld would not know which symbol is which).
I'd like to ask for suggestions on how to resolve this problem.
It's unfortunate that BerkeleyDB doesn't use a system like pkg-config, where the name of the library being linked is disconnected from the module being requested; if it were, we could build a libfeddb-6.so with symbol versioning and a libdb-6.so without, link our OS with the former by simply switching what the pkgconfig file points to, and maintain a semblance of binary compatibility with the outside world.
It's also unfortunate that symbol versioning falls over like this. The relevant section from Uli's writeup reads:
--- [... I]f the object with the references uses symbol versions but the object with the definitions has none. In this case a matching symbol is accepted unless the object's name matches the one in the Elfxx_Verneed entry. In the latter case this is a fatal error. In fact, this should never have happened in the first place since it would mean the list of required symbols was not correct and the steps required in the last section therefore haven't detected a too old version of an object file. ---
This sort of breaks the upgrade path from unversioned to versioned symbols within a library, if I understand it correctly; adding versions effectively requires bumping the object's soname, since otherwise linking against libfoo.so.1 just after versions were added would prevent that binary from running against an unversioned-but-otherwise-identical libfoo.so.1. (Unless the library still exports unversioned names for every symbol, I suppose.)
Which is doubly unfortunate, since it means ld can't just decide to enable --default-symver at some point, as every binary linked afterwards will be incompatible with prior runtimes.
I don't know if I have a good suggestion here, none of the alternatives sound appealing.
- ajax
Once upon a time, Adam Jackson ajax@redhat.com said:
I don't know if I have a good suggestion here, none of the alternatives sound appealing.
Is there a compelling reason to upgrade to BDB-6 (other than 6 > 5)? The license change really seems to be a PITA, since it is incompatible with the licenses of software already using BDB-5.
On 05/07/2014 10:21 PM, Chris Adams wrote:
Once upon a time, Adam Jackson ajax@redhat.com said:
I don't know if I have a good suggestion here, none of the alternatives sound appealing.
Is there a compelling reason to upgrade to BDB-6 (other than 6 > 5)? The license change really seems to be a PITA, since it is incompatible with the licenses of software already using BDB-5.
We should provide some libdb officially supported by upstream, which won't be the case for libdb-5 forever. Also there may come some projects in the future that will require libdb-6 for some of its features. So, staying only with libdb-5 does not seem to be a solution for me.
Honza
Once upon a time, Honza Horak hhorak@redhat.com said:
We should provide some libdb officially supported by upstream, which won't be the case for libdb-5 forever.
Is there a list yet of the packages using libdb-5 that have a license issue with libdb-6? If there are many, there may be community interest in forking libdb-5 (or just picking up maintenance if/when upstream drops it).
On Wed, May 07, 2014 at 03:19:42PM +0200, Jan Staněk wrote:
One of the planned parts of the F21 System Wide Change: BerkeleyDB 6 [1] is the introduction of downstream symbol versioning of both versions of the libraries (libdb with v6 and libdb5 with v5). This part is planned in order to not introduce bugs similar to [2]. However, if we introduce downstream versioning (as upstream is generally unresponsive), then we face the problem similar to [3].
Just wondering: is this getting punted to F22 now?
Regards, Joe
----- Original Message -----
On Wed, May 07, 2014 at 03:19:42PM +0200, Jan Staněk wrote:
One of the planned parts of the F21 System Wide Change: BerkeleyDB 6 [1] is the introduction of downstream symbol versioning of both versions of the libraries (libdb with v6 and libdb5 with v5). This part is planned in order to not introduce bugs similar to [2]. However, if we introduce downstream versioning (as upstream is generally unresponsive), then we face the problem similar to [3].
Just wondering: is this getting punted to F22 now?
The current plan is to not ship v6 in F21, and there are no agreed plans to ship it in any future release either. See https://fedorahosted.org/fesco/ticket/1291 for the full history of the debate. Mirek
On 07/22/2014 04:08 PM, Miloslav Trmač wrote:
The current plan is to not ship v6 in F21, and there are no agreed plans to ship it in any future release either. See https://fedorahosted.org/fesco/ticket/1291 for the full history of the debate.
Yet Oracle advertises Fedora's use of Berkeley DB in a recent release announcement:
“Open source Fedora package maintainer, Lubomir Rintel, says "Berkeley DB has quietly served behind the scenes as the database for the RPM Package Manager. It has proven itself time and time again as a robust and efficient storage engine. It stores the meta information of the installed rpms. Under heavy workloads, BDB proves itself reliable. Countless people that use popular Linux distributions have used BDB through RPM and never knew it. With this new release, BDB continues its tradition of being a solid storage engine"”
https://blogs.oracle.com/berkeleydb/entry/berkeley_db_12cr1_12_1
On 22 July 2014 09:05, Florian Weimer fweimer@redhat.com wrote:
On 07/22/2014 04:08 PM, Miloslav Trmač wrote:
The current plan is to not ship v6 in F21, and there are no agreed plans to ship it in any future release either. See https://fedorahosted.org/ fesco/ticket/1291 for the full history of the debate.
Yet Oracle advertises Fedora's use of Berkeley DB in a recent release announcement:
You are talking past each other. Oracle is talking about v5 in that release even if they are saying it in a v6 announcement. v6 is the one that Miloslav says isn't being looked at in future releases.
“Open source Fedora package maintainer, Lubomir Rintel, says "Berkeley DB has quietly served behind the scenes as the database for the RPM Package Manager. It has proven itself time and time again as a robust and efficient storage engine. It stores the meta information of the installed rpms. Under heavy workloads, BDB proves itself reliable. Countless people that use popular Linux distributions have used BDB through RPM and never knew it. With this new release, BDB continues its tradition of being a solid storage engine"”
https://blogs.oracle.com/berkeleydb/entry/berkeley_db_12cr1_12_1
-- Florian Weimer / Red Hat Product Security -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
On 22.7.2014 17:16, Stephen John Smoogen wrote:
On 22 July 2014 09:05, Florian Weimer fweimer@redhat.com wrote:
On 07/22/2014 04:08 PM, Miloslav Trmač wrote:
The current plan is to not ship v6 in F21, and there are no agreed plans to ship it in any future release either. See https://fedorahosted.org/ fesco/ticket/1291 for the full history of the debate.
Yet Oracle advertises Fedora's use of Berkeley DB in a recent release announcement:
You are talking past each other. Oracle is talking about v5 in that release even if they are saying it in a v6 announcement. v6 is the one that Miloslav says isn't being looked at in future releases.
I hope that one day we will get rid of BDB and move to MDB: https://bugzilla.redhat.com/show_bug.cgi?id=1086784
Besides other things it should have significantly better performance...
Petr^2 Spacek
“Open source Fedora package maintainer, Lubomir Rintel, says "Berkeley DB has quietly served behind the scenes as the database for the RPM Package Manager. It has proven itself time and time again as a robust and efficient storage engine. It stores the meta information of the installed rpms. Under heavy workloads, BDB proves itself reliable. Countless people that use popular Linux distributions have used BDB through RPM and never knew it. With this new release, BDB continues its tradition of being a solid storage engine"”
https://blogs.oracle.com/berkeleydb/entry/berkeley_db_12cr1_12_1
-
Adam Jackson -
Chris Adams -
Florian Weimer -
Honza Horak -
Jan Staněk -
Joe Orton -
Michael Stahl -
Miloslav Trmač -
Miloslav Trmač -
Petr Spacek -
Stephen John Smoogen