only in the x86_64 directories. Is this on purpose? Are the x86_64 images valid?
On Sun, 2009-04-05 at 13:38 -0400, Clyde E. Kunkel wrote:
On 04/04/2009 08:43 AM, Clyde E. Kunkel wrote:
only in the x86_64 directories. Is this on purpose? Are the x86_64 images valid?
Still no i386 images, and still x86_64 images.
Anyone know what is going on?
Should I ask on development list?
Today there were broken deps at the time of the compose. The last couple days there has been an odd rpmdb corruption issue on the compose chroots that I've been investigating.
On Mon, 6 Apr 2009, Jesse Keating wrote:
On Sun, 2009-04-05 at 13:38 -0400, Clyde E. Kunkel wrote:
On 04/04/2009 08:43 AM, Clyde E. Kunkel wrote:
only in the x86_64 directories. Is this on purpose? Are the x86_64 images valid?
Still no i386 images, and still x86_64 images.
Anyone know what is going on?
Should I ask on development list?
Today there were broken deps at the time of the compose. The last couple days there has been an odd rpmdb corruption issue on the compose chroots that I've been investigating.
Details please? Actual rpmdb *corruption* is rare, it's almost always something else. "Last couple of days" sounds like it could be the related to the NSS caused breakage.
- Panu -
On Mon, 2009-04-06 at 19:33 +0300, Panu Matilainen wrote:
Details please? Actual rpmdb *corruption* is rare, it's almost always something else. "Last couple of days" sounds like it could be the related to the NSS caused breakage.
When on an x86_64 system and trying to compose an i386 install tree, the buildinstall process falls over when trying to compose the image due to what looks like rpmdb corruption, but could be something else. Note that we explicitly rm -f the /var/lib/rpm/__db* files before running pungi (buildinstall). I am not able to reproduce outside our build environment, and even outside this particular host, yet. Still investigating.
http://kojipkgs.fedoraproject.org/mash/rawhide-20090403/logs/i386.log
Details please? Actual rpmdb *corruption* is rare, it's almost always something else. "Last couple of days" sounds like it could be the related to the NSS caused breakage.
When on an x86_64 system and trying to compose an i386 install tree, the buildinstall process falls over when trying to compose the image due to what looks like rpmdb corruption, but could be something else. Note that we explicitly rm -f the /var/lib/rpm/__db* files before running pungi (buildinstall). I am not able to reproduce outside our build environment, and even outside this particular host, yet. Still investigating.
http://kojipkgs.fedoraproject.org/mash/rawhide-20090403/logs/i386.log
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Peter
On Mon, 6 Apr 2009, Peter Robinson wrote:
Details please? Actual rpmdb *corruption* is rare, it's almost always something else. "Last couple of days" sounds like it could be the related to the NSS caused breakage.
When on an x86_64 system and trying to compose an i386 install tree, the buildinstall process falls over when trying to compose the image due to what looks like rpmdb corruption, but could be something else. Note that we explicitly rm -f the /var/lib/rpm/__db* files before running pungi (buildinstall). I am not able to reproduce outside our build environment, and even outside this particular host, yet. Still investigating.
http://kojipkgs.fedoraproject.org/mash/rawhide-20090403/logs/i386.log
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post the exact errors you're seeing and the command that produced it. Also, does rpm itself work (ie is it just yum that's busted) and if not, what errors you get from that?
- Panu -
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post the exact errors you're seeing and the command that produced it. Also, does rpm itself work (ie is it just yum that's busted) and if not, what errors you get from that?
Its just plain rpm. An example doing a simple package query gives me the following
[root@cypher ~]# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. error: cannot open Packages index using db3 - No such file or directory (2) error: Unterminated {: {_%{_dbpat� 0< / error: cannot open Packages database in / package glibc is not installed [root@cypher ~]#
Let me know if you want me to do further testing.
Peter
On Tue, 7 Apr 2009, Peter Robinson wrote:
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post the exact errors you're seeing and the command that produced it. Also, does rpm itself work (ie is it just yum that's busted) and if not, what errors you get from that?
Its just plain rpm. An example doing a simple package query gives me the following
[root@cypher ~]# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. error: cannot open Packages index using db3 - No such file or directory (2) error: Unterminated {: {_%{_dbpat??? 0< / error: cannot open Packages database in / package glibc is not installed [root@cypher ~]#
Let me know if you want me to do further testing.
Smells strongly of memory corruption... Can you run the same with valgrind?
# valgrind --leak-check=yes rpm -q glibc
Thanks,
- Panu -
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post the exact errors you're seeing and the command that produced it. Also, does rpm itself work (ie is it just yum that's busted) and if not, what errors you get from that?
Its just plain rpm. An example doing a simple package query gives me the following
[root@cypher ~]# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. error: cannot open Packages index using db3 - No such file or directory (2) error: Unterminated {: {_%{_dbpat??? 0< / error: cannot open Packages database in / package glibc is not installed [root@cypher ~]#
Let me know if you want me to do further testing.
Smells strongly of memory corruption... Can you run the same with valgrind?
# valgrind --leak-check=yes rpm -q glibc
Hmm. don't have valgrind installed and rpm is screwed so I can't..... suggestions?
Peter
On Tue, 7 Apr 2009, Peter Robinson wrote:
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post the exact errors you're seeing and the command that produced it. Also, does rpm itself work (ie is it just yum that's busted) and if not, what errors you get from that?
Its just plain rpm. An example doing a simple package query gives me the following
[root@cypher ~]# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. error: cannot open Packages index using db3 - No such file or directory (2) error: Unterminated {: {_%{_dbpat??? 0< / error: cannot open Packages database in / package glibc is not installed [root@cypher ~]#
Let me know if you want me to do further testing.
Smells strongly of memory corruption... Can you run the same with valgrind?
# valgrind --leak-check=yes rpm -q glibc
Hmm. don't have valgrind installed and rpm is screwed so I can't..... suggestions?
rpm2cpio should work even if rpm is otherwise screwed (and if even that fails, there's a script version in /usr/lib/rpm/rpm2cpio.sh). Also just a strace might provide some clues, if you have that installed.
- Panu -
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post the exact errors you're seeing and the command that produced it. Also, does rpm itself work (ie is it just yum that's busted) and if not, what errors you get from that?
Its just plain rpm. An example doing a simple package query gives me the following
[root@cypher ~]# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. error: cannot open Packages index using db3 - No such file or directory (2) error: Unterminated {: {_%{_dbpat??? 0< / error: cannot open Packages database in / package glibc is not installed [root@cypher ~]#
Let me know if you want me to do further testing.
Smells strongly of memory corruption... Can you run the same with valgrind?
# valgrind --leak-check=yes rpm -q glibc
Hmm. don't have valgrind installed and rpm is screwed so I can't..... suggestions?
rpm2cpio should work even if rpm is otherwise screwed (and if even that fails, there's a script version in /usr/lib/rpm/rpm2cpio.sh). Also just a strace might provide some clues, if you have that installed.
Any idea what provides the memcheck tool?
[root@cypher bin]# ./valgrind --leak-check=yes rpm -q glibc valgrind: failed to start tool 'memcheck' for platform 'x86-linux': No such file or directory [root@cypher bin]#
Peter
On Tue, 7 Apr 2009, Peter Robinson wrote:
> I'm getting that exact error on a Fit-PC machine (AMD Geode of similar > spec to the OLPC) that's running Fedora 10. I went to upgrade it on > the weekend to F11 beta. As part of the yum upgrade I did a "yum > upgrade rpm yum" which of course pulled in python. The geode is > capable of cmov so it has a i686 glibc installed (but it complains > about the arch) so rather than installing the i386 F10 glibc so it > would go smoothly I used rpm to install the F11 i586 glibc package. > Everything runs fine but I get the same errors with rpm so its stuck > in that state at the moment. I went to do a clean install of F11 but X > fell over (which worked on F10 install) so I haven't bothered to go > any further if you want me to run some tests.
Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post the exact errors you're seeing and the command that produced it. Also, does rpm itself work (ie is it just yum that's busted) and if not, what errors you get from that?
Its just plain rpm. An example doing a simple package query gives me the following
[root@cypher ~]# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. error: cannot open Packages index using db3 - No such file or directory (2) error: Unterminated {: {_%{_dbpat??? 0< / error: cannot open Packages database in / package glibc is not installed [root@cypher ~]#
Let me know if you want me to do further testing.
Smells strongly of memory corruption... Can you run the same with valgrind?
# valgrind --leak-check=yes rpm -q glibc
Hmm. don't have valgrind installed and rpm is screwed so I can't..... suggestions?
rpm2cpio should work even if rpm is otherwise screwed (and if even that fails, there's a script version in /usr/lib/rpm/rpm2cpio.sh). Also just a strace might provide some clues, if you have that installed.
Any idea what provides the memcheck tool?
[root@cypher bin]# ./valgrind --leak-check=yes rpm -q glibc valgrind: failed to start tool 'memcheck' for platform 'x86-linux': No such file or directory [root@cypher bin]#
valgrind expects to find the bits and pieces in certain places - you'll need to do the rpm2cpio unpacking from / to get the paths right.
- Panu -
> Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post > the > exact errors you're seeing and the command that produced it. Also, > does > rpm > itself work (ie is it just yum that's busted) and if not, what errors > you > get from that?
Its just plain rpm. An example doing a simple package query gives me the following
[root@cypher ~]# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. error: cannot open Packages index using db3 - No such file or directory (2) error: Unterminated {: {_%{_dbpat??? 0< / error: cannot open Packages database in / package glibc is not installed [root@cypher ~]#
Let me know if you want me to do further testing.
Smells strongly of memory corruption... Can you run the same with valgrind?
# valgrind --leak-check=yes rpm -q glibc
Hmm. don't have valgrind installed and rpm is screwed so I can't..... suggestions?
rpm2cpio should work even if rpm is otherwise screwed (and if even that fails, there's a script version in /usr/lib/rpm/rpm2cpio.sh). Also just a strace might provide some clues, if you have that installed.
Any idea what provides the memcheck tool?
[root@cypher bin]# ./valgrind --leak-check=yes rpm -q glibc valgrind: failed to start tool 'memcheck' for platform 'x86-linux': No such file or directory [root@cypher bin]#
valgrind expects to find the bits and pieces in certain places - you'll need to do the rpm2cpio unpacking from / to get the paths right.
Output can be found here. Let me know if you want any other things run.
http://pbrobinson.fedorapeople.org/rpm-valgrind.txt
Cheers, Peter
On Tue, 7 Apr 2009, Peter Robinson wrote:
>> Hmm, Geode doesn't fit the NSS-case as it's not multilib. Please post >> the >> exact errors you're seeing and the command that produced it. Also, >> does >> rpm >> itself work (ie is it just yum that's busted) and if not, what errors >> you >> get from that? > > Its just plain rpm. An example doing a simple package query gives me > the following > > [root@cypher ~]# rpm -q glibc > error: Unterminated {: {_%{_keyringpath}/*.k > 0< / > error: /: reading of public key failed. > error: cannot open Packages index using db3 - No such file or > directory > (2) > error: Unterminated {: {_%{_dbpat??? > 0< / > error: cannot open Packages database in / > package glibc is not installed > [root@cypher ~]# > > Let me know if you want me to do further testing.
Smells strongly of memory corruption... Can you run the same with valgrind?
# valgrind --leak-check=yes rpm -q glibc
Hmm. don't have valgrind installed and rpm is screwed so I can't..... suggestions?
rpm2cpio should work even if rpm is otherwise screwed (and if even that fails, there's a script version in /usr/lib/rpm/rpm2cpio.sh). Also just a strace might provide some clues, if you have that installed.
Any idea what provides the memcheck tool?
[root@cypher bin]# ./valgrind --leak-check=yes rpm -q glibc valgrind: failed to start tool 'memcheck' for platform 'x86-linux': No such file or directory [root@cypher bin]#
valgrind expects to find the bits and pieces in certain places - you'll need to do the rpm2cpio unpacking from / to get the paths right.
Output can be found here. Let me know if you want any other things run.
Hmm, this appears to be F10 valgrind (==3619== Using valgrind-3.3.0), which is fairly broken at least as far as rpm goes (it doesn't handle pipe2() syscall which prevents macros from getting properly processed), try grabbing rawhide valgrind instead and rerun.
Some related items: despite macros being somewhat busted, it actually manages to return the glibc version: glibc-2.9.90-12.i586. That is rawhide alright, but then:
==3623== by 0xBAB7C3: rpmInitCrypto (in /usr/lib/librpmio-4.6.so)
...it still has rpm 4.6.0, whereas rawhide has 4.7.0-beta1. That this happens with rpm 4.6.0 too is probably a useful piece of information. Since rpm seems to limp on somehow under valgrind (which is might be a clue in itself), try grabbing the entire package list with 'valgrind rpm -qa', maybe there are some further clues there. (and yes this is pretty much shooting in the dark, trying to hunt for some clues)
- Panu -
>> Its just plain rpm. An example doing a simple package query gives me >> the following >> >> [root@cypher ~]# rpm -q glibc >> error: Unterminated {: {_%{_keyringpath}/*.k >> 0< / >> error: /: reading of public key failed. >> error: cannot open Packages index using db3 - No such file or >> directory >> (2) >> error: Unterminated {: {_%{_dbpat??? >> 0< / >> error: cannot open Packages database in / >> package glibc is not installed >> [root@cypher ~]# >> >> Let me know if you want me to do further testing. > > Smells strongly of memory corruption... Can you run the same with > valgrind? > > # valgrind --leak-check=yes rpm -q glibc
Hmm. don't have valgrind installed and rpm is screwed so I can't..... suggestions?
rpm2cpio should work even if rpm is otherwise screwed (and if even that fails, there's a script version in /usr/lib/rpm/rpm2cpio.sh). Also just a strace might provide some clues, if you have that installed.
Any idea what provides the memcheck tool?
[root@cypher bin]# ./valgrind --leak-check=yes rpm -q glibc valgrind: failed to start tool 'memcheck' for platform 'x86-linux': No such file or directory [root@cypher bin]#
valgrind expects to find the bits and pieces in certain places - you'll need to do the rpm2cpio unpacking from / to get the paths right.
Output can be found here. Let me know if you want any other things run.
Hmm, this appears to be F10 valgrind (==3619== Using valgrind-3.3.0), which is fairly broken at least as far as rpm goes (it doesn't handle pipe2() syscall which prevents macros from getting properly processed), try grabbing rawhide valgrind instead and rerun.
Some related items: despite macros being somewhat busted, it actually manages to return the glibc version: glibc-2.9.90-12.i586. That is rawhide alright, but then:
==3623== by 0xBAB7C3: rpmInitCrypto (in /usr/lib/librpmio-4.6.so)
...it still has rpm 4.6.0, whereas rawhide has 4.7.0-beta1. That this happens with rpm 4.6.0 too is probably a useful piece of information. Since rpm seems to limp on somehow under valgrind (which is might be a clue in itself), try grabbing the entire package list with 'valgrind rpm -qa', maybe there are some further clues there. (and yes this is pretty much shooting in the dark, trying to hunt for some clues)
Basically the only thing on the box that's rawhide is glibc which is about where it got to before it barfed. If you contact me off list I can give you a login to the device to investigate if you like.
Peter
On Wed, 8 Apr 2009, Peter Robinson wrote:
>>> Its just plain rpm. An example doing a simple package query gives me >>> the following >>> >>> [root@cypher ~]# rpm -q glibc >>> error: Unterminated {: {_%{_keyringpath}/*.k >>> 0< / >>> error: /: reading of public key failed. >>> error: cannot open Packages index using db3 - No such file or >>> directory >>> (2) >>> error: Unterminated {: {_%{_dbpat??? >>> 0< / >>> error: cannot open Packages database in / >>> package glibc is not installed >>> [root@cypher ~]# >>> >>> Let me know if you want me to do further testing. >> >> Smells strongly of memory corruption... Can you run the same with >> valgrind? >> >> # valgrind --leak-check=yes rpm -q glibc > > Hmm. don't have valgrind installed and rpm is screwed so I can't..... > suggestions?
rpm2cpio should work even if rpm is otherwise screwed (and if even that fails, there's a script version in /usr/lib/rpm/rpm2cpio.sh). Also just a strace might provide some clues, if you have that installed.
Any idea what provides the memcheck tool?
[root@cypher bin]# ./valgrind --leak-check=yes rpm -q glibc valgrind: failed to start tool 'memcheck' for platform 'x86-linux': No such file or directory [root@cypher bin]#
valgrind expects to find the bits and pieces in certain places - you'll need to do the rpm2cpio unpacking from / to get the paths right.
Output can be found here. Let me know if you want any other things run.
Hmm, this appears to be F10 valgrind (==3619== Using valgrind-3.3.0), which is fairly broken at least as far as rpm goes (it doesn't handle pipe2() syscall which prevents macros from getting properly processed), try grabbing rawhide valgrind instead and rerun.
Some related items: despite macros being somewhat busted, it actually manages to return the glibc version: glibc-2.9.90-12.i586. That is rawhide alright, but then:
==3623== by 0xBAB7C3: rpmInitCrypto (in /usr/lib/librpmio-4.6.so)
...it still has rpm 4.6.0, whereas rawhide has 4.7.0-beta1. That this happens with rpm 4.6.0 too is probably a useful piece of information. Since rpm seems to limp on somehow under valgrind (which is might be a clue in itself), try grabbing the entire package list with 'valgrind rpm -qa', maybe there are some further clues there. (and yes this is pretty much shooting in the dark, trying to hunt for some clues)
Basically the only thing on the box that's rawhide is glibc which is about where it got to before it barfed.
Ahhah, this might be an all-important clue: * Wed Apr 1 2009 Jakub Jelinek jakub@redhat.com 2.9.90-12 - update from trunk - configure with --enable-experimental-malloc
Note "experimental malloc", Apr 1st is the first known occurrence of what appears to be memory corruption, and except for glibc, you have essentially F10 system exhibiting this issue. It's at least a rather promising lead if nothing else.
If you contact me off list I can give you a login to the device to investigate if you like.
Thanks, I might contact you later depending on where the trail leads :)
- Panu -
Panu Matilainen wrote:
Ahhah, this might be an all-important clue:
- Wed Apr 1 2009 Jakub Jelinek jakub@redhat.com 2.9.90-12
- update from trunk
- configure with --enable-experimental-malloc
Note "experimental malloc", Apr 1st is the first known occurrence of what appears to be memory corruption, and except for glibc, you have essentially F10 system exhibiting this issue. It's at least a rather promising lead if nothing else.
What can be funnier for April Fools than breaking RPM? ^^
Kevin Kofler
On Wed, 2009-04-08 at 11:29 +0300, Panu Matilainen wrote:
Basically the only thing on the box that's rawhide is glibc which is about where it got to before it barfed.
Ahhah, this might be an all-important clue:
- Wed Apr 1 2009 Jakub Jelinek jakub@redhat.com 2.9.90-12
- update from trunk
- configure with --enable-experimental-malloc
Note "experimental malloc", Apr 1st is the first known occurrence of what appears to be memory corruption, and except for glibc, you have essentially F10 system exhibiting this issue. It's at least a rather promising lead if nothing else.
If you contact me off list I can give you a login to the device to investigate if you like.
Thanks, I might contact you later depending on where the trail leads :)
I didn't see a bug filed for this issue already. I've filed the following bug to track this issue.
Bug#494867 - "Rawhide i386 fails to boot - not syncing: No init found."
It's currently assigned to the 'rpm' component, but I suspect that will change with further analysis.
Thanks, James
On Wed, 8 Apr 2009, James Laska wrote:
On Wed, 2009-04-08 at 11:29 +0300, Panu Matilainen wrote:
Basically the only thing on the box that's rawhide is glibc which is about where it got to before it barfed.
Ahhah, this might be an all-important clue:
- Wed Apr 1 2009 Jakub Jelinek jakub@redhat.com 2.9.90-12
- update from trunk
- configure with --enable-experimental-malloc
Note "experimental malloc", Apr 1st is the first known occurrence of what appears to be memory corruption, and except for glibc, you have essentially F10 system exhibiting this issue. It's at least a rather promising lead if nothing else.
If you contact me off list I can give you a login to the device to investigate if you like.
Thanks, I might contact you later depending on where the trail leads :)
I didn't see a bug filed for this issue already. I've filed the following bug to track this issue.
Bug#494867 - "Rawhide i386 fails to boot - not syncing: No init found."
It's currently assigned to the 'rpm' component, but I suspect that will change with further analysis.
FYI, it's glibc-2.9.90-12 i586-version that's broken, i686 version works just fine. And there's apparently a bug in yum that makes it pick i586 over i686 on some situations. Thanks to Jesse Keating for hunting it down.
- Panu -
Basically the only thing on the box that's rawhide is glibc which is about where it got to before it barfed.
Ahhah, this might be an all-important clue:
- Wed Apr 1 2009 Jakub Jelinek jakub@redhat.com 2.9.90-12
- update from trunk
- configure with --enable-experimental-malloc
Note "experimental malloc", Apr 1st is the first known occurrence of what appears to be memory corruption, and except for glibc, you have essentially F10 system exhibiting this issue. It's at least a rather promising lead if nothing else.
If you contact me off list I can give you a login to the device to investigate if you like.
Thanks, I might contact you later depending on where the trail leads :)
I didn't see a bug filed for this issue already. I've filed the following bug to track this issue.
Bug#494867 - "Rawhide i386 fails to boot - not syncing: No init found."
It's currently assigned to the 'rpm' component, but I suspect that will change with further analysis.
FYI, it's glibc-2.9.90-12 i586-version that's broken, i686 version works just fine. And there's apparently a bug in yum that makes it pick i586 over i686 on some situations. Thanks to Jesse Keating for hunting it down.
So is it fixed in rawhide now or is it still pending?
Peter
On Thu, 9 Apr 2009, Peter Robinson wrote:
Basically the only thing on the box that's rawhide is glibc which is about where it got to before it barfed.
Ahhah, this might be an all-important clue:
- Wed Apr 1 2009 Jakub Jelinek jakub@redhat.com 2.9.90-12
- update from trunk
- configure with --enable-experimental-malloc
Note "experimental malloc", Apr 1st is the first known occurrence of what appears to be memory corruption, and except for glibc, you have essentially F10 system exhibiting this issue. It's at least a rather promising lead if nothing else.
If you contact me off list I can give you a login to the device to investigate if you like.
Thanks, I might contact you later depending on where the trail leads :)
I didn't see a bug filed for this issue already. I've filed the following bug to track this issue.
Bug#494867 - "Rawhide i386 fails to boot - not syncing: No init found."
It's currently assigned to the 'rpm' component, but I suspect that will change with further analysis.
FYI, it's glibc-2.9.90-12 i586-version that's broken, i686 version works just fine. And there's apparently a bug in yum that makes it pick i586 over i686 on some situations. Thanks to Jesse Keating for hunting it down.
So is it fixed in rawhide now or is it still pending?
glibc-2.9.90-14 has the experimental malloc disabled but apparently that is not the problem, i586 glibc is still broken:
bash-4.0# rpm -Uvh glibc-common-2.9.90-14.i586.rpm glibc-2.9.90-14.i586.rpm error: failed to open /etc/mtab: No such file or directory Preparing... ########################################### [100%] 1:glibc ########################################### [ 50%] 2:glibc-common ########################################### [100%] bash-4.0# rpm -q glibc error: Unterminated {: {_%{_keyringpath}/*.k 0< / error: /: reading of public key failed. package glibc is not installed
- Panu -
On Mon, 6 Apr 2009, Peter Robinson wrote:
Details please? Actual rpmdb *corruption* is rare, it's almost always something else. "Last couple of days" sounds like it could be the related to the NSS caused breakage.
When on an x86_64 system and trying to compose an i386 install tree, the buildinstall process falls over when trying to compose the image due to what looks like rpmdb corruption, but could be something else. Note that we explicitly rm -f the /var/lib/rpm/__db* files before running pungi (buildinstall). I am not able to reproduce outside our build environment, and even outside this particular host, yet. Still investigating.
http://kojipkgs.fedoraproject.org/mash/rawhide-20090403/logs/i386.log
I'm getting that exact error on a Fit-PC machine (AMD Geode of similar spec to the OLPC) that's running Fedora 10. I went to upgrade it on the weekend to F11 beta. As part of the yum upgrade I did a "yum upgrade rpm yum" which of course pulled in python. The geode is capable of cmov so it has a i686 glibc installed (but it complains about the arch) so rather than installing the i386 F10 glibc so it would go smoothly I used rpm to install the F11 i586 glibc package. Everything runs fine but I get the same errors with rpm so its stuck in that state at the moment. I went to do a clean install of F11 but X fell over (which worked on F10 install) so I haven't bothered to go any further if you want me to run some tests.
Oh and btw, "rpm --showrc" and strace of the failing run might provide some clues.
- Panu -
On Mon, 6 Apr 2009, Jesse Keating wrote:
On Mon, 2009-04-06 at 19:33 +0300, Panu Matilainen wrote:
Details please? Actual rpmdb *corruption* is rare, it's almost always something else. "Last couple of days" sounds like it could be the related to the NSS caused breakage.
When on an x86_64 system and trying to compose an i386 install tree, the buildinstall process falls over when trying to compose the image due to what looks like rpmdb corruption, but could be something else. Note that we explicitly rm -f the /var/lib/rpm/__db* files before running pungi (buildinstall). I am not able to reproduce outside our build environment, and even outside this particular host, yet. Still investigating.
http://kojipkgs.fedoraproject.org/mash/rawhide-20090403/logs/i386.log
i386 on x86_64 borking could easily fit the recent NSS thingie, which is exactly a multilib issue (bug 494122, nss-softokn-freebl getting pulled in for just one, possibly wrong arch on multilib systems). See if it goes away with nss-3.12.2.99.3-7.
The errors look like memory corruption of sorts (garbled/truncated macro strings), if that goes away by updating NSS then it's probably some path(s) not correctly checking for failures from NSS (there are probably plenty) and tripping up something else in turn.
- Panu -
On Mon, 2009-04-06 at 20:46 +0300, Panu Matilainen wrote:
http://kojipkgs.fedoraproject.org/mash/rawhide-20090403/logs/i386.log
i386 on x86_64 borking could easily fit the recent NSS thingie, which is exactly a multilib issue (bug 494122, nss-softokn-freebl getting pulled in for just one, possibly wrong arch on multilib systems). See if it goes away with nss-3.12.2.99.3-7.
I really don't think that's the case, unless somehow something went so wrong that you're getting an x86_64 package in an i386 chroot. We're using an x86_64 host to generate an i386 chroot, so all the content inside the chroot should be i386 (well i386/i586/i686 but you get the picture)
The errors look like memory corruption of sorts (garbled/truncated macro strings), if that goes away by updating NSS then it's probably some path(s) not correctly checking for failures from NSS (there are probably plenty) and tripping up something else in turn.
Updating NSS where? These chroots are being made from the koji static-repos for dist-f11, is there a newer nss that is not even in koji?
On Mon, 6 Apr 2009, Jesse Keating wrote:
On Mon, 2009-04-06 at 20:46 +0300, Panu Matilainen wrote:
http://kojipkgs.fedoraproject.org/mash/rawhide-20090403/logs/i386.log
i386 on x86_64 borking could easily fit the recent NSS thingie, which is exactly a multilib issue (bug 494122, nss-softokn-freebl getting pulled in for just one, possibly wrong arch on multilib systems). See if it goes away with nss-3.12.2.99.3-7.
I really don't think that's the case, unless somehow something went so wrong that you're getting an x86_64 package in an i386 chroot. We're using an x86_64 host to generate an i386 chroot, so all the content inside the chroot should be i386 (well i386/i586/i686 but you get the picture)
The problem would be on the x86_64 host generating the root, if it were to be the NSS-issue. But ok I suppose the host isn't running rawhide... So I suppose it's something completely different, I just fail to see what could've caused such a thing in the last couple of days.
The errors look like memory corruption of sorts (garbled/truncated macro strings), if that goes away by updating NSS then it's probably some path(s) not correctly checking for failures from NSS (there are probably plenty) and tripping up something else in turn.
Updating NSS where? These chroots are being made from the koji static-repos for dist-f11, is there a newer nss that is not even in koji?
The latest NSS on koji and supposed to be in next rawhide compose but ... if Peter is seeing similar errors on Geode then it's really something quite different. Maybe something limited to ix86, I wouldn't notice as all my boxes are x86_64. Getting late here though, I'll try to see if I can somehow reproduce it tomorrow. Rawhide seems to really be living up to its name at the moment...
- Panu -
On Mon, 2009-04-06 at 21:57 +0300, Panu Matilainen wrote:
The latest NSS on koji and supposed to be in next rawhide compose but ... if Peter is seeing similar errors on Geode then it's really something quite different. Maybe something limited to ix86, I wouldn't notice as all my boxes are x86_64. Getting late here though, I'll try to see if I can somehow reproduce it tomorrow. Rawhide seems to really be living up to its name at the moment...
This may be specific to the rpm version used to produce the chroot in the first place, then rawhide usage within the chroot. I haven't been able to reproduce here locally using rawhide to generate the rawhide chroot. The rpm used in the failing scenario is rpm-4.6.0-4.0.mitr.1.el5.x86_64 on EL5.