Signed-off-by: Steven Dake sdake@redhat.com --- man/oz-install.1 | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/man/oz-install.1 b/man/oz-install.1 index b497813..7cf0077 100644 --- a/man/oz-install.1 +++ b/man/oz-install.1 @@ -70,9 +70,14 @@ will undefine the libvirt guest with the same name or UUID and delete the diskimage, so it should be used with caution. .TP .B "-t" -Use a timeout value of \fBtimeout\fR for installation, rather than the -oz default. This can be useful if you know you have slower storage -and want to wait longer for the installation to timeout. +Terminate the installation of the guest image in \fBtimeout\fR seconds +rather then the default of 1200 seconds. This value can be increased in the +case of slow storage or multiple oz-install operations on the same machine +consuming the disk bandwidth. + +Please note there is a separate termination action that occurs if 300 seconds +elapses before any data is written to the VM image. This timer value is not +configurable. .TP .B "-u" Customize the image after installation. This generally installs
The default 300 second timer is insufficient to install RHEL6.1 on a W510 laptop. When the installation terminates with the 300 second default, the install.img file is only 70% read from disk on this system.
Signed-off-by: Steven Dake sdake@redhat.com --- oz/Guest.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/oz/Guest.py b/oz/Guest.py index a0d1385..23e403e 100644 --- a/oz/Guest.py +++ b/oz/Guest.py @@ -388,7 +388,7 @@ class Guest(object): f.truncate(size * 1024 * 1024 * 1024) f.close()
- def wait_for_install_finish(self, libvirt_dom, count, inactivity_timeout=300): + def wait_for_install_finish(self, libvirt_dom, count, inactivity_timeout=1000): # first find the disk device we are installing to; this will be # monitored for activity during the installation domxml = libvirt_dom.XMLDesc(0)
On 06/08/11 - 09:25:48PM, Steven Dake wrote:
The default 300 second timer is insufficient to install RHEL6.1 on a W510 laptop. When the installation terminates with the 300 second default, the install.img file is only 70% read from disk on this system.
Hm, can you be a bit more clear here? Is this still during the initial anaconda stage where it is reading the install.img from the CD-ROM? If that is the case, I might suggest that instead of increasing the timeout, we also monitor the CD device for activity. That is, if either the diskimage or the CD-ROM has activity, we continue waiting.
On Thu, Jun 09, 2011 at 08:18:17AM -0400, Chris Lalancette wrote:
On 06/08/11 - 09:25:48PM, Steven Dake wrote:
The default 300 second timer is insufficient to install RHEL6.1 on a W510 laptop. When the installation terminates with the 300 second default, the install.img file is only 70% read from disk on this system.
Hm, can you be a bit more clear here? Is this still during the initial anaconda stage where it is reading the install.img from the CD-ROM? If that is the case, I might suggest that instead of increasing the timeout, we also monitor the CD device for activity. That is, if either the diskimage or the CD-ROM has activity, we continue waiting.
Hi guys, but i was thinking if is not a bit smarter instead of monitor the CDROM or such stuff, to have an Observer of the installation progress somehow. In that way, you should not be worried to change the timeout, since it is auto driven.
My 2cents, Francesco Vollero
-- Chris Lalancette _______________________________________________ aeolus-devel mailing list aeolus-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel
On 06/09/11 - 02:23:29PM, Francesco Vollero wrote:
On Thu, Jun 09, 2011 at 08:18:17AM -0400, Chris Lalancette wrote:
On 06/08/11 - 09:25:48PM, Steven Dake wrote:
The default 300 second timer is insufficient to install RHEL6.1 on a W510 laptop. When the installation terminates with the 300 second default, the install.img file is only 70% read from disk on this system.
Hm, can you be a bit more clear here? Is this still during the initial anaconda stage where it is reading the install.img from the CD-ROM? If that is the case, I might suggest that instead of increasing the timeout, we also monitor the CD device for activity. That is, if either the diskimage or the CD-ROM has activity, we continue waiting.
Hi guys, but i was thinking if is not a bit smarter instead of monitor the CDROM or such stuff, to have an Observer of the installation progress somehow. In that way, you should not be worried to change the timeout, since it is auto driven.
Yes, but that is unfortunately very difficult to do. For instance, prior to Fedora-14, there is really no good way to get logs out of the virtual machine anaconda to watch it. Even if you *do* get the logs out of the virtual machine, the output is not really machine parseable, so you have to resort to string parsing.
And even if you do all of that, you still need a timeout. How do you know that stage 12 of the anaconda install process has frozen, or is just taking a long time to complete?
On 06/09/2011 05:18 AM, Chris Lalancette wrote:
On 06/08/11 - 09:25:48PM, Steven Dake wrote:
The default 300 second timer is insufficient to install RHEL6.1 on a W510 laptop. When the installation terminates with the 300 second default, the install.img file is only 70% read from disk on this system.
Hm, can you be a bit more clear here? Is this still during the initial anaconda stage where it is reading the install.img from the CD-ROM? If that is the case, I might suggest that instead of increasing the timeout, we also monitor the CD device for activity. That is, if either the diskimage or the CD-ROM has activity, we continue waiting.
yes it is during the anaconda load of the image. Your proposal makes sense - I was going to take a crack at making something like that, but didn't really have any idea how to do that :) As is today the value is too low though.
Regards -steve
On 06/09/11 - 07:53:55AM, Steven Dake wrote:
On 06/09/2011 05:18 AM, Chris Lalancette wrote:
On 06/08/11 - 09:25:48PM, Steven Dake wrote:
The default 300 second timer is insufficient to install RHEL6.1 on a W510 laptop. When the installation terminates with the 300 second default, the install.img file is only 70% read from disk on this system.
Hm, can you be a bit more clear here? Is this still during the initial anaconda stage where it is reading the install.img from the CD-ROM? If that is the case, I might suggest that instead of increasing the timeout, we also monitor the CD device for activity. That is, if either the diskimage or the CD-ROM has activity, we continue waiting.
yes it is during the anaconda load of the image. Your proposal makes sense - I was going to take a crack at making something like that, but didn't really have any idea how to do that :) As is today the value is too low though.
Here's the problem with increasing the timeout:
The whole reason I introduced this extra complexity was to reduce the amount of time people needed to wait on a failed instance (think crashed kernel, installer, etc). If we bump this timeout to 1000, we've eliminated most of that usefulness and we might as well just remove the complexity and drop back to a standard timeout.
Before doing that, though, I think I would like to try watching the CD-ROM for activity. If that does the trick, I'd much rather go that route than increasing the timeout.
I'll whip up a patch for you to test tomorrow.
Thanks,
On 06/09/2011 12:48 PM, Chris Lalancette wrote:
On 06/09/11 - 07:53:55AM, Steven Dake wrote:
On 06/09/2011 05:18 AM, Chris Lalancette wrote:
On 06/08/11 - 09:25:48PM, Steven Dake wrote:
The default 300 second timer is insufficient to install RHEL6.1 on a W510 laptop. When the installation terminates with the 300 second default, the install.img file is only 70% read from disk on this system.
Hm, can you be a bit more clear here? Is this still during the initial anaconda stage where it is reading the install.img from the CD-ROM? If that is the case, I might suggest that instead of increasing the timeout, we also monitor the CD device for activity. That is, if either the diskimage or the CD-ROM has activity, we continue waiting.
yes it is during the anaconda load of the image. Your proposal makes sense - I was going to take a crack at making something like that, but didn't really have any idea how to do that :) As is today the value is too low though.
Here's the problem with increasing the timeout:
The whole reason I introduced this extra complexity was to reduce the amount of time people needed to wait on a failed instance (think crashed kernel, installer, etc). If we bump this timeout to 1000, we've eliminated most of that usefulness and we might as well just remove the complexity and drop back to a standard timeout.
Before doing that, though, I think I would like to try watching the CD-ROM for activity. If that does the trick, I'd much rather go that route than increasing the timeout.
I'll whip up a patch for you to test tomorrow.
sounds good i'm happy to test it out.
Regards -steve
Thanks,
On 06/09/11 - 01:09:17PM, Steven Dake wrote:
Before doing that, though, I think I would like to try watching the CD-ROM for activity. If that does the trick, I'd much rather go that route than increasing the timeout.
I'll whip up a patch for you to test tomorrow.
sounds good i'm happy to test it out.
Attached is a patch which I think should do the trick. At least, it should be monitoring the CD for activity as well as the destination disk image. Let me know if this improves the situation for you.
Thanks,
On 06/10/2011 07:29 AM, Chris Lalancette wrote:
On 06/09/11 - 01:09:17PM, Steven Dake wrote:
Before doing that, though, I think I would like to try watching the CD-ROM for activity. If that does the trick, I'd much rather go that route than increasing the timeout.
I'll whip up a patch for you to test tomorrow.
sounds good i'm happy to test it out.
Attached is a patch which I think should do the trick. At least, it should be monitoring the CD for activity as well as the destination disk image. Let me know if this improves the situation for you.
Thanks,
Same problem occurs. I'll spend a bit of time debugging to see if BlockStats is behaving as expected by the patch and see if I can sort something out.
aeolus-devel mailing list aeolus-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel
On 06/10/2011 09:00 AM, Steven Dake wrote:
On 06/10/2011 07:29 AM, Chris Lalancette wrote:
On 06/09/11 - 01:09:17PM, Steven Dake wrote:
Before doing that, though, I think I would like to try watching the CD-ROM for activity. If that does the trick, I'd much rather go that route than increasing the timeout.
I'll whip up a patch for you to test tomorrow.
sounds good i'm happy to test it out.
Attached is a patch which I think should do the trick. At least, it should be monitoring the CD for activity as well as the destination disk image. Let me know if this improves the situation for you.
Thanks,
Same problem occurs. I'll spend a bit of time debugging to see if BlockStats is behaving as expected by the patch and see if I can sort something out.
I added a debug log when the following values are set. They don't appear to change after the image install starts even as it is "counting up" from 0 to 100%. The first is the first device (vda) the second is the second device (hdc).
rd_req 334 rd_bytes 1368064 rd_req 1168 rd_bytes 138235904
The value of rd_bytes starts at 0, then increases a bit during the install process, then jumps to this 13mb value when the image loading starts.
Regards -steve
aeolus-devel mailing list aeolus-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel
aeolus-devel mailing list aeolus-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel
On 06/10/11 - 10:03:16AM, Steven Dake wrote:
On 06/10/2011 09:00 AM, Steven Dake wrote:
On 06/10/2011 07:29 AM, Chris Lalancette wrote:
On 06/09/11 - 01:09:17PM, Steven Dake wrote:
Before doing that, though, I think I would like to try watching the CD-ROM for activity. If that does the trick, I'd much rather go that route than increasing the timeout.
I'll whip up a patch for you to test tomorrow.
sounds good i'm happy to test it out.
Attached is a patch which I think should do the trick. At least, it should be monitoring the CD for activity as well as the destination disk image. Let me know if this improves the situation for you.
Thanks,
Same problem occurs. I'll spend a bit of time debugging to see if BlockStats is behaving as expected by the patch and see if I can sort something out.
I added a debug log when the following values are set. They don't appear to change after the image install starts even as it is "counting up" from 0 to 100%. The first is the first device (vda) the second is the second device (hdc).
rd_req 334 rd_bytes 1368064 rd_req 1168 rd_bytes 138235904
The value of rd_bytes starts at 0, then increases a bit during the install process, then jumps to this 13mb value when the image loading starts.
Yeah, OK. As we discussed on IRC, this is probably because you are doing a URL based install from a far-away location, and it is taking more than 5 minutes to fetch the second anaconda stage. The attached patch monitors the network for activity as well; can you give it a whirl?
Thanks,
On 06/10/2011 12:39 PM, Chris Lalancette wrote:
On 06/10/11 - 10:03:16AM, Steven Dake wrote:
On 06/10/2011 09:00 AM, Steven Dake wrote:
On 06/10/2011 07:29 AM, Chris Lalancette wrote:
On 06/09/11 - 01:09:17PM, Steven Dake wrote:
Before doing that, though, I think I would like to try watching the CD-ROM for activity. If that does the trick, I'd much rather go that route than increasing the timeout.
I'll whip up a patch for you to test tomorrow.
sounds good i'm happy to test it out.
Attached is a patch which I think should do the trick. At least, it should be monitoring the CD for activity as well as the destination disk image. Let me know if this improves the situation for you.
Thanks,
Same problem occurs. I'll spend a bit of time debugging to see if BlockStats is behaving as expected by the patch and see if I can sort something out.
I added a debug log when the following values are set. They don't appear to change after the image install starts even as it is "counting up" from 0 to 100%. The first is the first device (vda) the second is the second device (hdc).
rd_req 334 rd_bytes 1368064 rd_req 1168 rd_bytes 138235904
The value of rd_bytes starts at 0, then increases a bit during the install process, then jumps to this 13mb value when the image loading starts.
Yeah, OK. As we discussed on IRC, this is probably because you are doing a URL based install from a far-away location, and it is taking more than 5 minutes to fetch the second anaconda stage. The attached patch monitors the network for activity as well; can you give it a whirl?
Thanks,
No go on this patch - this version fails:
oz.OzException.OzException: No disk or network activity in 300 seconds, failing
On 06/10/11 - 01:11:12PM, Steven Dake wrote:
Yeah, OK. As we discussed on IRC, this is probably because you are doing a URL based install from a far-away location, and it is taking more than 5 minutes to fetch the second anaconda stage. The attached patch monitors the network for activity as well; can you give it a whirl?
Thanks,
No go on this patch - this version fails:
oz.OzException.OzException: No disk or network activity in 300 seconds, failing
All right. Instead of poking around in the dark, the attached patch monitors both the network and all disk devices, and also spews debug data about activity that we are seeing. Can you try this patch, and send me the output? The output will most likely be very large, so I suggest redirecting stdout to a file and then just mailing me the file directly.
Thanks
I think I might have hit the timeout with 0.98.15.z3 of iwhd. See log below for details. Is this a regression or was a patch/method never adopted?
[root@cldmgr01 ~]# rpm -qi iwhd Name : iwhd Relocations: (not relocatable) Version : 0.98.15.z3 Vendor: (none) Release : 1.el6 Build Date: Tue 16 Aug 2011 02:08:09 PM EDT Install Date: Tue 16 Aug 2011 02:42:47 PM EDT Build Host: simbelmyne.zaitcev.lan Group : System Environment/Libraries Source RPM: iwhd-0.98.15.z3-1.el6.src.rpm Size : 427672 License: GPLv3 Signature : (none) URL : http://git.fedorahosted.org/git/?p=iwhd.git Summary : Image WareHouse Daemon Description : Deltacloud image-warehouse daemon
[root@cldmgr01 ~]# tail -f /var/log/imagefactory.log <target dev="hdc"/> </disk> </devices> </domain>
2011-10-18 05:06:59,255 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3600/3600 2011-10-18 05:07:09,282 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3590/3600 2011-10-18 05:07:19,305 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3580/3600 2011-10-18 05:07:29,329 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3570/3600 2011-10-18 05:07:39,353 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3560/3600 2011-10-18 05:07:49,376 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3550/3600 2011-10-18 05:07:59,401 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3540/3600 2011-10-18 05:08:09,428 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3530/3600 2011-10-18 05:08:19,451 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3520/3600 2011-10-18 05:08:29,475 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3510/3600 2011-10-18 05:08:39,499 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3500/3600 2011-10-18 05:08:49,526 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3490/3600 2011-10-18 05:08:59,550 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3480/3600 2011-10-18 05:09:09,578 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3470/3600 2011-10-18 05:09:19,603 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3460/3600 2011-10-18 05:09:29,629 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3450/3600 2011-10-18 05:09:39,655 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3440/3600 2011-10-18 05:09:49,675 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3430/3600 2011-10-18 05:09:59,694 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3420/3600 2011-10-18 05:10:09,719 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3410/3600 2011-10-18 05:10:19,743 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3400/3600 2011-10-18 05:10:29,763 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3390/3600 2011-10-18 05:10:39,785 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3380/3600 2011-10-18 05:10:49,815 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3370/3600 2011-10-18 05:10:59,840 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3360/3600 2011-10-18 05:11:09,861 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3350/3600 2011-10-18 05:11:19,887 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3340/3600 2011-10-18 05:11:29,914 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3330/3600 2011-10-18 05:11:39,945 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3320/3600 2011-10-18 05:11:49,974 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3310/3600 2011-10-18 05:11:59,999 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3300/3600 2011-10-18 05:12:10,030 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3290/3600 2011-10-18 05:12:20,056 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3280/3600 2011-10-18 05:12:30,083 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3270/3600 2011-10-18 05:12:40,111 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3260/3600 2011-10-18 05:12:50,140 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3250/3600 2011-10-18 05:13:00,164 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3240/3600 2011-10-18 05:13:10,198 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3230/3600 2011-10-18 05:13:20,224 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3220/3600 2011-10-18 05:13:30,251 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3210/3600 2011-10-18 05:13:40,280 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Waiting for tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 to finish installing, 3200/3600 2011-10-18 05:13:45,446 DEBUG imagefactory.builders.BaseBuilder.FedoraBuilder pid(3586) Message: Exception caught in ImageFactory 2011-10-18 05:13:45,447 DEBUG imagefactory.builders.BaseBuilder.FedoraBuilder pid(3586) Message: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/imagefactory/builders/FedoraBuilder.py", line 172, in build_upload libvirt_xml = self.guest.install(self.app_config["timeout"]) File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 1154, in install return self.do_install(timeout, force, 0) File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 1139, in do_install self.wait_for_install_finish(dom, timeout) File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 451, in wait_for_install_finish raise oz.OzException.OzException("No disk activity in %d seconds, failing" % (inactivity_timeout)) OzException: No disk activity in 300 seconds, failing
2011-10-18 05:13:45,447 INFO oz.Guest.FedoraGuest pid(3586) Message: Cleaning up guest named tmpl1-36f6669e-7f0f-44b5-8ab3-41c66daddeb8 2011-10-18 05:13:45,944 DEBUG imagefactory.BuildJob.BuildAdaptor pid(3586) Message: Raising event with agent handler (<ImageFactoryAgent(Thread-1, initial)>), changed status from BUILDING to FAILED 2011-10-18 05:13:45,945 INFO oz.Guest.FedoraGuest pid(3586) Message: Cleaning up after install 2011-10-18 05:13:45,945 DEBUG oz.Guest.FedoraGuest pid(3586) Message: Removing modified ISO 2011-10-18 05:13:45,983 DEBUG imagefactory.builders.BaseBuilder.FedoraBuilder pid(3586) Message: Exception caught in ImageFactory 2011-10-18 05:13:45,984 DEBUG imagefactory.builders.BaseBuilder.FedoraBuilder pid(3586) Message: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/imagefactory/builders/FedoraBuilder.py", line 124, in build_image self.build_upload(build_id) File "/usr/lib/python2.6/site-packages/imagefactory/builders/FedoraBuilder.py", line 172, in build_upload libvirt_xml = self.guest.install(self.app_config["timeout"]) File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 1154, in install return self.do_install(timeout, force, 0) File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 1139, in do_install self.wait_for_install_finish(dom, timeout) File "/usr/lib/python2.6/site-packages/oz/Guest.py", line 451, in wait_for_install_finish raise oz.OzException.OzException("No disk activity in %d seconds, failing" % (inactivity_timeout)) OzException: No disk activity in 300 seconds, failing
2011-10-18 05:13:45,985 DEBUG imagefactory.BuildJob.BuildAdaptor pid(3586) Message: Raising event with agent handler (<ImageFactoryAgent(Thread-1, initial)>), changed status from FAILED to FAILED
----- Original Message -----
From: "Chris Lalancette" clalance@redhat.com To: "Steven Dake" sdake@redhat.com Cc: aeolus-devel@lists.fedorahosted.org, "Chris Lalancette" clalance@redhat.com Sent: Tuesday, June 14, 2011 10:58:23 AM Subject: Re: [PATCH 2/3] Increase hard-coded 300 second timer to 1000 seconds.
On 06/10/11 - 01:11:12PM, Steven Dake wrote:
Yeah, OK. As we discussed on IRC, this is probably because you are doing a URL based install from a far-away location, and it is taking more than 5 minutes to fetch the second anaconda stage. The attached patch monitors the network for activity as well; can you give it a whirl?
Thanks,
No go on this patch - this version fails:
oz.OzException.OzException: No disk or network activity in 300 seconds, failing
All right. Instead of poking around in the dark, the attached patch monitors both the network and all disk devices, and also spews debug data about activity that we are seeing. Can you try this patch, and send me the output? The output will most likely be very large, so I suggest redirecting stdout to a file and then just mailing me the file directly.
Thanks
Chris Lalancette
aeolus-devel mailing list aeolus-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel
On 10/18/11 - 10:19:05AM, James Labocki wrote:
I think I might have hit the timeout with 0.98.15.z3 of iwhd. See log below for details. Is this a regression or was a patch/method never adopted?
I did commit this patch, but it was to Oz. What version of Oz do you have installed?
One thing that is on the TODO list, but which I never did, was to add a screenshot capture of this situation, which could help to debug here. One of these days :).
# rpm -qi oz Name : oz Relocations: (not relocatable) Version : 0.5.0 Vendor: (none) Release : 1.el6 Build Date: Fri 08 Jul 2011 01:00:18 PM EDT Install Date: Wed 27 Jul 2011 04:48:19 AM EDT Build Host: aeolus-fedora.virt.bos.redhat.com Group : Development/Libraries Source RPM: oz-0.5.0-1.el6.src.rpm Size : 429852 License: LGPLv2 Signature : (none) URL : http://aeolusproject.org/oz.html Summary : Library and utilities for automated guest OS installs Description : Oz is a set of libraries and utilities for doing automated guest OS installations, with minimal input from the user.
I found the screenshot (in /) and it looks like some package metadata is missing. Let me fix that.
----- Original Message -----
From: "Chris Lalancette" clalance@redhat.com To: "James Labocki" jlabocki@redhat.com Cc: aeolus-devel@lists.fedorahosted.org, "Chris Lalancette" clalance@redhat.com, "Steven Dake" sdake@redhat.com Sent: Tuesday, October 18, 2011 11:25:36 AM Subject: Re: [PATCH 2/3] Increase hard-coded 300 second timer to 1000 seconds.
On 10/18/11 - 10:19:05AM, James Labocki wrote:
I think I might have hit the timeout with 0.98.15.z3 of iwhd. See log below for details. Is this a regression or was a patch/method never adopted?
I did commit this patch, but it was to Oz. What version of Oz do you have installed?
One thing that is on the TODO list, but which I never did, was to add a screenshot capture of this situation, which could help to debug here. One of these days :).
-- Chris Lalancette _______________________________________________ aeolus-devel mailing list aeolus-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel
On 10/18/11 - 01:37:52PM, James Labocki wrote:
# rpm -qi oz Name : oz Relocations: (not relocatable) Version : 0.5.0 Vendor: (none) Release : 1.el6 Build Date: Fri 08 Jul 2011 01:00:18 PM EDT Install Date: Wed 27 Jul 2011 04:48:19 AM EDT Build Host: aeolus-fedora.virt.bos.redhat.com Group : Development/Libraries Source RPM: oz-0.5.0-1.el6.src.rpm Size : 429852 License: LGPLv2 Signature : (none) URL : http://aeolusproject.org/oz.html Summary : Library and utilities for automated guest OS installs Description : Oz is a set of libraries and utilities for doing automated guest OS installations, with minimal input from the user.
I found the screenshot (in /) and it looks like some package metadata is missing. Let me fix that.
Oh yeah, and that Oz is ancient. I would highly suggest updating to the latest, which is Oz 0.7.0-3.
Looks like I just had to restart libvirtd (something got hosed) and the VM could then get to the repository. It looks like I have to wait until Oz 0.6 to be reminded to check the screenshot.
https://bugzilla.redhat.com/show_bug.cgi?id=726107
-James
----- Original Message -----
From: "James Labocki" jlabocki@redhat.com To: "Chris Lalancette" clalance@redhat.com Cc: aeolus-devel@lists.fedorahosted.org, "Steven Dake" sdake@redhat.com Sent: Tuesday, October 18, 2011 1:37:52 PM Subject: Re: [PATCH 2/3] Increase hard-coded 300 second timer to 1000 seconds.
# rpm -qi oz Name : oz Relocations: (not relocatable) Version : 0.5.0 Vendor: (none) Release : 1.el6 Build Date: Fri 08 Jul 2011 01:00:18 PM EDT Install Date: Wed 27 Jul 2011 04:48:19 AM EDT Build Host: aeolus-fedora.virt.bos.redhat.com Group : Development/Libraries Source RPM: oz-0.5.0-1.el6.src.rpm Size : 429852 License: LGPLv2 Signature : (none) URL : http://aeolusproject.org/oz.html Summary : Library and utilities for automated guest OS installs Description : Oz is a set of libraries and utilities for doing automated guest OS installations, with minimal input from the user.
I found the screenshot (in /) and it looks like some package metadata is missing. Let me fix that.
----- Original Message -----
From: "Chris Lalancette" clalance@redhat.com To: "James Labocki" jlabocki@redhat.com Cc: aeolus-devel@lists.fedorahosted.org, "Chris Lalancette" clalance@redhat.com, "Steven Dake" sdake@redhat.com Sent: Tuesday, October 18, 2011 11:25:36 AM Subject: Re: [PATCH 2/3] Increase hard-coded 300 second timer to 1000 seconds.
On 10/18/11 - 10:19:05AM, James Labocki wrote:
I think I might have hit the timeout with 0.98.15.z3 of iwhd. See log below for details. Is this a regression or was a patch/method never adopted?
I did commit this patch, but it was to Oz. What version of Oz do you have installed?
One thing that is on the TODO list, but which I never did, was to add a screenshot capture of this situation, which could help to debug here. One of these days :).
-- Chris Lalancette _______________________________________________ aeolus-devel mailing list aeolus-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/aeolus-devel
Signed-off-by: Steven Dake sdake@redhat.com --- man/oz-install.1 | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/man/oz-install.1 b/man/oz-install.1 index 7cf0077..6d7b19f 100644 --- a/man/oz-install.1 +++ b/man/oz-install.1 @@ -75,7 +75,7 @@ rather then the default of 1200 seconds. This value can be increased in the case of slow storage or multiple oz-install operations on the same machine consuming the disk bandwidth.
-Please note there is a separate termination action that occurs if 300 seconds +Please note there is a separate termination action that occurs if 1000 seconds elapses before any data is written to the VM image. This timer value is not configurable. .TP
On 06/08/11 - 09:25:49PM, Steven Dake wrote:
Signed-off-by: Steven Dake sdake@redhat.com
man/oz-install.1 | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/man/oz-install.1 b/man/oz-install.1 index 7cf0077..6d7b19f 100644 --- a/man/oz-install.1 +++ b/man/oz-install.1 @@ -75,7 +75,7 @@ rather then the default of 1200 seconds. This value can be increased in the case of slow storage or multiple oz-install operations on the same machine consuming the disk bandwidth.
-Please note there is a separate termination action that occurs if 300 seconds +Please note there is a separate termination action that occurs if 1000 seconds elapses before any data is written to the VM image. This timer value is not configurable. .TP
This one will of course have to wait on the outcome of our discussion on patches 1 and 2 :).
Hey Steve,
Thanks for the patches.
On 06/08/11 - 09:25:47PM, Steven Dake wrote:
Signed-off-by: Steven Dake sdake@redhat.com
man/oz-install.1 | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/man/oz-install.1 b/man/oz-install.1 index b497813..7cf0077 100644 --- a/man/oz-install.1 +++ b/man/oz-install.1 @@ -70,9 +70,14 @@ will undefine the libvirt guest with the same name or UUID and delete the diskimage, so it should be used with caution. .TP .B "-t" -Use a timeout value of \fBtimeout\fR for installation, rather than the -oz default. This can be useful if you know you have slower storage -and want to wait longer for the installation to timeout. +Terminate the installation of the guest image in \fBtimeout\fR seconds +rather then the default of 1200 seconds. This value can be increased in the
I've tried to some extent not to mention what the defaults here are. The reason being that the defaults are different depending on what operating system you are installing. For Linux guests, the default is indeed 1200, but for Windows guests the default is 3600.
+case of slow storage or multiple oz-install operations on the same machine +consuming the disk bandwidth.
+Please note there is a separate termination action that occurs if 300 seconds +elapses before any data is written to the VM image. This timer value is not +configurable.
Right, this part makes sense. So maybe if we did a combination, and did something like:
Terminate the installation of the guest in \fBtimeout\fR seconds rather than the oz default. This value can be increased in the case of slow storage or multiple oz-install operations on the same machine.
Please note that there is a separate termination action that occurs if 300 seconds elapses before any data is written to the VM image. This timer value is not configurable.
Does that sound reasonable?
On 06/09/11 - 08:15:46AM, Chris Lalancette wrote:
Right, this part makes sense. So maybe if we did a combination, and did something like:
Terminate the installation of the guest in \fBtimeout\fR seconds rather than the oz default. This value can be increased in the case of slow storage or multiple oz-install operations on the same machine.
Please note that there is a separate termination action that occurs if 300 seconds elapses before any data is written to the VM image. This timer value is not configurable.
I've pushed a patch to the repository that looks something like the above text. Let me know if that is good with you. Thanks for the patch!
On 09/06/2011, at 10:15 PM, Chris Lalancette wrote:
+Terminate the installation of the guest image in \fBtimeout\fR seconds +rather then the default of 1200 seconds. This value can be increased in the
I've tried to some extent not to mention what the defaults here are. The reason being that the defaults are different depending on what operating system you are installing. For Linux guests, the default is indeed 1200, but for Windows guests the default is 3600.
Hmmm, it's probably better to specifically mention them, so people are not caught unawares. Linux = 1200 seconds, windows = 3600 seconds, etc.
Regards and best wishes,
Justin Clift
-- Aeolus Community Manager http://www.aeolusproject.org
aeolus-devel@lists.fedorahosted.org