Hello Jim and David,

Thank you very much for your advice, the problem is solved!!!!
There is no rpm hangup bug.

After David's note about rpm -Va I have noticed that it takes a 'while' to finish and after David's question
> How long did you give it before nuking it ?
I have decided that I will be VERY patient. I have left rpm running overnight and in the morning it was finished.
The problem is that I do not know the exact time. My last record from top (before I get to bed) is:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
13632 root      25   0  238m 149m 4256 R 98.4 29.8  80:04.16 rpm

It took minimally more than an hour on the Pentium M 1.4Mhz eating all CPU... I have never seen rpm performing such a long time before.

Thank you very much for your help!!!

Have a nice weekend,

Jan
PS: I never delete files manually. I was in a wrong believe that rpm database is cleared of xen packages after xen entries for FC4 were deleted
      from the grub.conf after FC4->FC5t3 migration, so I decided co cleanup my system and I deleted xen kernels manually (but nothing else).

PPS: Here is rpm command I used and complete verbose output:

# rpm -evv --justdb kernel-xen0-devel kernel-xenU-devel --allmatches
D: opening  db environment /var/lib/rpm/Packages create:cdb:mpool
D: opening  db index       /var/lib/rpm/Packages rdonly mode=0x0
D: locked   db index       /var/lib/rpm/Packages
D: opening  db index       /var/lib/rpm/Name rdonly mode=0x0
D:  read h#     468 Header SHA1 digest: OK (99fd52c1d2a31ea6661ce0d249a26852b471d211)
D: opening  db index       /var/lib/rpm/Pubkeys rdonly mode=0x0
D:  read h#    1040 Header sanity check: OK
D: ========== DSA pubkey id b44269d0 4f2a6fd2 (h#1040)
D:  read h#     732 Header V3 DSA signature: OK, key ID 4f2a6fd2
D:  read h#     117 Header SHA1 digest: OK (c8d069d827dc6bd4e6293ceca8f1f45568480cfc)
D:  read h#     678 Header V3 DSA signature: OK, key ID 4f2a6fd2
D: ========== --- kernel-xen0-devel-2.6.12-1.1454_FC4 i686/linux 0x0
D: opening  db index       /var/lib/rpm/Requirename rdonly mode=0x0
D: ========== --- kernel-xen0-devel-2.6.13-1.1532_FC4 i686/linux 0x0
D: ========== --- kernel-xenU-devel-2.6.12-1.1454_FC4 i686/linux 0x0
D: ========== --- kernel-xenU-devel-2.6.13-1.1532_FC4 i686/linux 0x0
D: ========== recording tsort relations
D: ========== tsorting packages (order, #predecessors, #succesors, tree, depth, breadth)
D:     0    0    0    3    1    0   -kernel-xenU-devel-2.6.13-1.1532_FC4.i686
D: ========== successors only (0 bytes)
D:     1    0    0    0    1    1   - kernel-xen0-devel-2.6.12-1.1454_FC4.i686
D:     2    0    0    1    1    2   -kernel-xen0-devel-2.6.13-1.1532_FC4.i686
D:     3    0    0    2    1    3   -kernel-xenU-devel-2.6.12-1.1454_FC4.i686
D: closed   db index       /var/lib/rpm/Pubkeys
D: closed   db index       /var/lib/rpm/Requirename
D: closed   db index       /var/lib/rpm/Name
D: closed   db index       /var/lib/rpm/Packages
D: closed   db environment /var/lib/rpm/Packages
D: opening  db environment /var/lib/rpm/Packages joinenv
D: opening  db index       /var/lib/rpm/Packages create mode=0x42
D: mounted filesystems:
D:     i        dev    bsize       bavail       iavail mount point
D:     0 0x00000307     1024       286906       167116 /
D:     1 0x00000003     4096            0           -1 /proc
D:     2 0x00000000     4096            0           -1 /sys
D:     3 0x0000000a     4096            0           -1 /dev/pts
D:     4 0x00000011     4096        64439        64438 /dev/shm
D:     5 0x0000030c     1024       121091        98975 /home
D:     6 0x0000030b     1024       397617       131515 /tmp
D:     7 0x00000308     4096       252679      1462737 /usr
D:     8 0x0000030a     4096       120759       285237 /var
D:     9 0x00000013     4096            0           -1 /proc/sys/fs/binfmt_misc
D:    10 0x00000014     4096            0           -1 /var/lib/nfs/rpc_pipefs
D:    11 0x00000015     4096            0           -1 /net
D: sanity checking 4 elements
D: running pre-transaction scripts
D: computing 24608 file fingerprints
D: computing file dispositions
D: opening  db index       /var/lib/rpm/Basenames create mode=0x42
D: ========== --- kernel-xenU-devel-2.6.13-1.1532_FC4 i686-linux 0x0
D:     erase: kernel-xenU-devel-2.6.13-1.1532_FC4 has 5012 files, test = 0
D: opening  db index       /var/lib/rpm/Name create mode=0x42
D:  read h#     678 Header V3 DSA signature: OK, key ID 4f2a6fd2
D:   --- h#     678 kernel-xenU-devel-2.6.13-1.1532_FC4
D: removing "kernel-xenU-devel" from Name index.
D: removing 5012 entries from Basenames index.
D: opening  db index       /var/lib/rpm/Group create mode=0x42
D: removing "System Environment/Kernel" from Group index.
D: opening  db index       /var/lib/rpm/Requirename create mode=0x42
D: removing 6 entries from Requirename index.
D: opening  db index       /var/lib/rpm/Providename create mode=0x42
D: removing 4 entries from Providename index.
D: opening  db index       /var/lib/rpm/Dirnames create mode=0x42
D: removing 1475 entries from Dirnames index.
D: opening  db index       /var/lib/rpm/Requireversion create mode=0x42
D: removing 6 entries from Requireversion index.
D: opening  db index       /var/lib/rpm/Provideversion create mode=0x42
D: removing 4 entries from Provideversion index.
D: opening  db index       /var/lib/rpm/Installtid create mode=0x42
D: removing 1 entries from Installtid index.
D: opening  db index       /var/lib/rpm/Sigmd5 create mode=0x42
D: removing 1 entries from Sigmd5 index.
D: opening  db index       /var/lib/rpm/Sha1header create mode=0x42
D: removing "851a2f954dfbb73ffdca2e99746214e79edb8089" from Sha1header index.
D: opening  db index       /var/lib/rpm/Filemd5s create mode=0x42
D: removing 5012 entries from Filemd5s index.
D: ========== --- kernel-xen0-devel-2.6.13-1.1532_FC4 i686-linux 0x0
D:     erase: kernel-xen0-devel-2.6.13-1.1532_FC4 has 7414 files, test = 0
D:  read h#     732 Header V3 DSA signature: OK, key ID 4f2a6fd2
D:   --- h#     732 kernel-xen0-devel-2.6.13-1.1532_FC4
D: removing "kernel-xen0-devel" from Name index.
D: removing 7414 entries from Basenames index.
D: removing "System Environment/Kernel" from Group index.
D: removing 6 entries from Requirename index.
D: removing 4 entries from Providename index.
D: removing 2554 entries from Dirnames index.
D: removing 6 entries from Requireversion index.
D: removing 4 entries from Provideversion index.
D: removing 1 entries from Installtid index.
D: removing 1 entries from Sigmd5 index.
D: removing "3b91e9a818dae6f182b83d80d001b1605017dedc" from Sha1header index.
D: removing 7414 entries from Filemd5s index.
D: ========== --- kernel-xen0-devel-2.6.12-1.1454_FC4 i686-linux 0x0
D:     erase: kernel-xen0-devel-2.6.12-1.1454_FC4 has 7292 files, test = 0
D:  read h#     468 Header SHA1 digest: OK (99fd52c1d2a31ea6661ce0d249a26852b471d211)
D:   --- h#     468 kernel-xen0-devel-2.6.12-1.1454_FC4
D: removing "kernel-xen0-devel" from Name index.
D: removing 7292 entries from Basenames index.
D: removing "System Environment/Kernel" from Group index.
D: removing 6 entries from Requirename index.
D: removing 4 entries from Providename index.
D: removing 2513 entries from Dirnames index.
D: removing 6 entries from Requireversion index.
D: removing 4 entries from Provideversion index.
D: removing 1 entries from Installtid index.
D: removing 1 entries from Sigmd5 index.
D: removing "99fd52c1d2a31ea6661ce0d249a26852b471d211" from Sha1header index.
D: removing 7292 entries from Filemd5s index.
D: ========== --- kernel-xenU-devel-2.6.12-1.1454_FC4 i686-linux 0x0
D:     erase: kernel-xenU-devel-2.6.12-1.1454_FC4 has 4890 files, test = 0
D:  read h#     117 Header SHA1 digest: OK (c8d069d827dc6bd4e6293ceca8f1f45568480cfc)
D:   --- h#     117 kernel-xenU-devel-2.6.12-1.1454_FC4
D: removing "kernel-xenU-devel" from Name index.
D: removing 4890 entries from Basenames index.
D: removing "System Environment/Kernel" from Group index.
D: removing 6 entries from Requirename index.
D: removing 4 entries from Providename index.
D: removing 1434 entries from Dirnames index.
D: removing 6 entries from Requireversion index.
D: removing 4 entries from Provideversion index.
D: removing 1 entries from Installtid index.
D: removing 1 entries from Sigmd5 index.
D: removing "c8d069d827dc6bd4e6293ceca8f1f45568480cfc" from Sha1header index.
D: removing 4890 entries from Filemd5s index.
D: running post-transaction scripts
D: closed   db index       /var/lib/rpm/Filemd5s
D: closed   db index       /var/lib/rpm/Sha1header
D: closed   db index       /var/lib/rpm/Sigmd5
D: closed   db index       /var/lib/rpm/Installtid
D: closed   db index       /var/lib/rpm/Provideversion
D: closed   db index       /var/lib/rpm/Requireversion
D: closed   db index       /var/lib/rpm/Dirnames
D: closed   db index       /var/lib/rpm/Providename
D: closed   db index       /var/lib/rpm/Requirename
D: closed   db index       /var/lib/rpm/Group
D: closed   db index       /var/lib/rpm/Basenames
D: closed   db index       /var/lib/rpm/Name
D: closed   db index       /var/lib/rpm/Packages
D: closed   db environment /var/lib/rpm/Packages
D: May free Score board((nil))

Have a look on the Time+ value - 80:04.16 and it was not finished at this point...
On 3/4/06, Jim Cornette < fct-cornette@insight.rr.com> wrote:
Jan Andrejkovic wrote:
> Hello Jim and everybody who is willing to help,
>
> Runlevel 1 did not help, but I have found the source of the problem,
> unfortunatelly not the solution:
>
> The source of the problem is that I have deleted some installed files
> manually, without RPM.

Not a good practice. This is not wise for any operating system. RPM does
a great job removing and installing files when the rpm file is properly
referenced for installation and removal of packages. Not all packages
are always setup correctly but the bulk of rpms are well thought out by
those that setup the routines that rpm undergoes. There are many options
that rpm can handle and many that I am still not familiar with after
years of usage.

> Anyway I think rpm should be robust and it should not hang if some files
> are deleted.

It depends upon which files that you deleted. If you deleted a common
file or a file which is intricate to the program's operation or that of
the system, it will have no choice but to bomb out. Its primary jobs are
to allow adding removing of programs in a sane manner, ensuring that the
program contains all its needed components (binaries, setup files,
services started, user added, and on) It can only do so if all of the
components are left intact.
Since rpm checks for files being in a location, missing or that the file
is not like the orginal, it does catch the most likely situations. But
if rpm checked every file for its presence before trying to remove a
file, it would require a longer time for rpm to complete the task of
adding or removing the files.
It sounds like you might have pressed rpm and the package that you were
trying to remove into a corner case and if you can recall what you
deleted, it might be helpful to prevent a future incident in the
package, rpm program or user awareness and familiarity with the program
concept.

> It should display some warning instead.

Granted, rpm should ideally use its feature for verifying everything is
present before attempting a package removal. It should at least check
for the file presence or have a routine to accomplish if the file was
missing. I believe most rpms are setup to continue or report the problem
to the terminal output if a problem was encountered.
There are problems that happen when failures happen with %post and %pre
scripts that need improvement in design. The major drawbacks are when
%pre scripts fail and programs are setup to be installed, they may not
ever install because deps are checked before the installation
transaction. Things seemed setup for the transaction to do the right
thing. The problem might be that the package is setup correctly, but
security programs may not grant permission to rpm to place files, run
tasks and other unpredictable problems.
The problem with %post script failures is that the tasks might be
completed pretty much and there was only a minor failure because the rpm
was on its cleanup phase. Other %post failures might cause operational
tasks from completing such as a kernel never getting through to the
portion where the image needed for the modules and the entry into the
bootloader never gets completed. I am sure that the packagers keep all
of these factors in mind and do their best to prevent problems from
occurring.

>
> I have used -vv option and here is deailed output from rpm:
> rm -f /var/lib/rpm/__db*

the deleted portion looks normal to me. I however never packages a
program for distribution.

> D:     0    0    0    3    1    0   -
> kernel-xenU-devel-2.6.13-1.1532_FC4.i686
> D: ========== successors only (0 bytes)

Those that know more about the workings of rpm would know what the six
variables represent. The display of the rpm output does reveal to me
something more than I previously knew about rpm internals. I take it
that the program found no previous version to act upon. I am only going
on the zero bytes feedback.

> D:     1    0    0    0    1    1
> -kernel-xen0-devel-2.6.12-1.1454_FC4.i686
> D:     2    0    0    1    1    2
> -kernel-xen0-devel-2.6.13-1.1532_FC4.i686
> D:     3    0    0    2    1    3
> -kernel-xenU-devel-2.6.12-1.1454_FC4.i686

It looks like the first of six digits is some counter (0,1,2,3)


> D: closed   db index       /var/lib/rpm/Pubkeys
> D: closed   db index       /var/lib/rpm/Requirename
> D: closed   db index       /var/lib/rpm/Name
> D: closed   db index       /var/lib/rpm/Packages
> D: closed   db environment /var/lib/rpm/Packages
> D: opening  db environment /var/lib/rpm/Packages joinenv
> D: opening  db index       /var/lib/rpm/Packages create mode=0x42
> D: mounted filesystems:
> D:     i        dev    bsize       bavail       iavail mount point
> D:     0 0x00000307     1024       287812       167116 /
> D:     1 0x00000003     4096            0           -1 /proc
> D:     2 0x00000000     4096            0           -1 /sys
> D:     3 0x0000000a     4096            0           -1 /dev/pts
> D:     4 0x00000011     4096        64439        64438 /dev/shm
> D:     5 0x0000030c     1024       135546        99333 /home
> D:     6 0x0000030b     1024       397619       131519 /tmp
> D:     7 0x00000308     4096       260065      1462823 /usr
> D:     8 0x0000030a     4096       157409       285427 /var
> D:     9 0x00000013     4096            0           -1
> /proc/sys/fs/binfmt_misc
> D:    10 0x00000014     4096            0           -1
> /var/lib/nfs/rpc_pipefs
> D:    11 0x00000015     4096            0           -1 /net
> D: sanity checking 4 elements

Rpm and the package being acted upon are trying to be sane.

> D: running pre-transaction scripts

This I take is the %pre factor mentioned earlier. What do I need to do
before proceeding with the installation of the package?

> D: computing 24608 file fingerprints
> D: computing file dispositions
> D: opening  db index       /var/lib/rpm/Basenames create mode=0x42
> Killed

You might be able to use a program that can actually peek into the
actual package to see what the following step would be. There is a file
manager utility/shell program called mc (midnight commander) which is
able to dive into the rpm content. The script is executable so be sure
to use the F3 function key to view the file content. Being able to view
what is actually within a packages rpm might show interest to you and
enlighten you more than the link you referenced tries to explain.
I have used mc to view rpms and deb package content and found the
internals of both formats interesting. It is also possible to retrieve
files from the rpms, deb or whatever files with mc.

>
> It hanged after the last step and I had to kill it.

If the similarity is anything like DOS loading stuff, it is usually the
step following the lockup. It locked and did not reveal anything
regarding its initialization. Personally, I like feedback before
performing the function and feedback as to what the results were for the
tasks.

> I have straced it as well and here is the part of output from strace
> which is running in a neverending loop:
> <cut>
> stat64("/usr/src/kernels/2.6.12- 1.1454_FC4-xenU-i686/arch/i386",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12- 1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-
> 1.1454_FC4-xenU-i686/arch/i386/mach-visws", 0xbfbd6784) = -1 ENOENT (No
> such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12- 1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/mach-voyager",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/math-emu",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12- 1.1454_FC4-xenU-i686/arch/i386",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/mm",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386 ",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12- 1.1454_FC4-xenU-i686/arch/i386/oprofile",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/pci",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12- 1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> = -1 ENOENT (No such file or directory)
> stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/power",
> 0xbfbd6784) = -1 ENOENT (No such file or directory)
> <cut>
>
> I have deleted my FC4 xen kernels in FC5t3 manually because I thought
> that rpm dabase is already clear. I was wrong.
> My RPM version is: 4.4.2
>
> Do you think I should open bugzilla ticket?

It is a shortcoming of the package, not rpm to be able to handle this
circumstance. I file bug reports myself for the developer to at least be
aware of something that could be handled better. Most reports are marked
as not supported or the like. The awareness for the developer is at
least presented to the developer. Backburner or not, the rpm routine
should be able to handle this possible situation.

> Do you have any advice how can I reinstall my xen or what can I do?

rpm has features where you can ignore scripts, delete only the rpmdb
entry and a host of possible resolutions. Rpm most likely has features
which will get you out of the problem.

I would run
rpm -e --justdb --nodeps <continuous-loop-package>
  and then try to *install* this rpm again if desired. By reviewing the
output above, running my favorite command above to rid the rpmdb entry
from the older version package should be enough. Reading your original
posting, it seems that you already attempted this feature.

>
> By the way I have found very good rpm guide, but it did not help me either:
> http://www.redhat.com/docs/books/max-rpm

I'll have to take a look at this page later myself. I never read the
documentation before.
>
> Thank you very much,
>
> Jan
>
> On 3/1/06, *Jim Cornette * < fct-cornette@insight.rr.com
> <mailto:fct-cornette@insight.rr.com>> wrote:
>
>     Jan Andrejkovic wrote:
>      > Hello,
>      >
>      > I have upgraded FC4 to FC5t3. I had some xen problems therefore I
>     have
>      > decided to reinstall xen packages.
>      > But when I try
>      > rpm -e kernel-xen-hypervisor-devel or rpm -e
>     kernel-xen-guest-devel rpm
>      > hangs up and eats almost 100% cpu for long time.
>      > I need to use kill -9 to stop it.

Were you removing 1454 or 1532. If you have multiple versions installed,
specifying which version might be needed. RPM should have bombed out
specifying that multiple versions installed and to specify which version
that you wanted to remove.

>      >
>      > I have tried to do
>      > rm -f /var/lib/rpm/__db*
>      > and
>      > rpm --rebuilddb
>      > but nothing helped after those commands - it hangs again.
>      >
>      > I have also tried rpm -e --justdb but it did not help either.

Without specifying the version to remove? Or specifying which verson to
remove?

>    >

>> Initially it was "yum remove" which hanged first. But as far as I
> know it uses rpm therefore I report this as rpm problem.


I have no idea for certain. I missed a lot of information from your
original posting as reference to the steps you tried.

>>
>> Does anybody have the same problem or can anybody tell me some
>> workaround how I can remove those packages manually and possibly
> rebuild database without them?

I believe the database does not need rebuilt. The entries within the
database need corrected by human intervention using the features rpm is
capable of.

I hope that I am not misleading or jumping off on unrelated to your
problem. I doubt that is the case though, I probably departed on several
issues. I was afraid to reply to the posting initially with all the
information provided in verbose and debugging output.

I'd figure out how best to represent the problem with yum and the
hypervisor removal and your actions. I believe you attempted using yum
then rpm and later deleted the files afterward. It sounds like a really
big bug since the looping and cpu utilization.

>>
>> Thank you very much,
>>
>> Jan
>
>

I hope this leads you to a resolution of the problem.

Jim

--
What I want is all of the power and none of the responsibility.

--
fedora-test-list mailing list
fedora-test-list@redhat.com
To unsubscribe:
https://www.redhat.com/mailman/listinfo/fedora-test-list