On Mon 12 Aug 2013 04:13:26 AM CEST, Zhou Zheng Sheng wrote:
Hi David,
on 2013-08-09 21:45, David Caro Estevez wrote:
Sometimes we get this error when running the vdsm tests:
====================================================================== ERROR: testLoopMount (mountTests.MountTests)
Traceback (most recent call last): File "/ephemeral0/vdsm_unit_tests_gerrit_el/tests/mountTests.py", line 69, in testLoopMount m.mount(mntOpts="loop") File "/ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py", line 222, in mount return self._runcmd(cmd, timeout) File "/ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py", line 238, in _runcmd raise MountError(rc, ";".join((out, err))) MountError: (2, ';mount: could not find any free loop device\n') -------------------- >> begin captured logging << -------------------- Storage.Misc.excCmd: DEBUG: '/sbin/mkfs.ext2 -F /tmp/tmpq95svr' (cwd None) Storage.Misc.excCmd: DEBUG: SUCCESS: <err> = 'mke2fs 1.41.12 (17-May-2010)\n'; <rc> = 0 Storage.Misc.excCmd: DEBUG: '/usr/bin/sudo -n /bin/mount -o loop /tmp/tmpq95svr /tmp/tmpcS29EU' (cwd None)
The problem is that it seems that the loop devices that are not being released (maybe when the test fails?) and the system runs out of devices eventually. Can you take a look to see where the cleanup fails and fix it?
I think this may be related to a known bug [1] of gvfs.
To workaround the bug,
- Reboot Linux
- See if "ps aux | grep gvfs" produces any results, then kill all gvfs
related processes. 3. "cd tests", and run the tests "./run_tests_local.sh mkimageTests.py mountTests.py" for 10 times, and "losetup -a" should give empty result after each run. When gvfs is running, "losetup -a" would show 2 new loop devices are occupied after each run.
gvfs is not installed on the machines, so I doubt this is the same error :( I suggest to add some cleanup code at the end of the tests to free all the used loop devices (if you free only the ones you use, it will not fail when running parallel tests).
Adding more loop devices or cleaning them all manually is only a temporary solution.
Thanks!
-- David Caro
Red Hat Czech s.r.o. Continuous Integration Engineer - EMEA ENG Virtualization R&D
Tel.: +420 532 294 605 Email: dcaro@redhat.com Web: www.cz.redhat.com Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno, Czech Republic RHT Global #: 82-62605