Test that broken upstream
by smizrahi@redhat.com
Upstream will not currently build because of the following patch:
commit 1d4f220616ca6fc014bbdfef7b826a16ed608ddf
Author: y kaplan <ykaplan(a)redhat.com>
Date: Thu Apr 5 18:02:40 2012 +0300
Added guestIFTests
Change-Id: I8b5138296c098826f149c26d38fd2bfce8794fe4
Reviewed-on: http://gerrit.ovirt.org/3379
Reviewed-by: Dan Kenigsberg <danken(a)redhat.com>
Tested-by: Dan Kenigsberg <danken(a)redhat.com>
It's import utils before setting up constants.
The reason it worked for the committer is that it had VDSM installed on the host so it could import vdsm from site-packages.
Please note that when writing tests and try to test with a clean host.
We are in the process of having Jenkins plug in to Gerrit to do it for us before we actually commit and break the build. But until then please double check.
12 years, 2 months
Is there a preferred workflow when adding additional error messages to VDSM _and_ Engine?
by lyarwood@redhat.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello all,
I'm currently working on a patch to correct the logic used by vdsm when
reviewing the logical and physical block size of physical volumes [1].
Once/If this is approved I'd like to look into making the exceptions
thrown from vdsm and the errors displayed within engine more descriptive.
Before I start throwing patches into gerrit I was wondering *if* there
is a preferred workflow when adding additional exceptions and error
messages across both projects. For example, committing changes to vdsm
before engine, engine before vdsm or both at the same time.
I'm also wondering if there is a way to say that vdsm gerrit changeid x
relies on engine gerrit changeid y etc?
Thanks in advance,
Lee
- --
Lee Yarwood
Software Maintenance Engineer
Red Hat UK Ltd
200 Fowler Avenue IQ Farnborough, Farnborough, Hants GU14 7JP
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham (USA), Brendan Lane (Ireland), Matt
Parson(USA), Charlie Peters (USA)
GPG fingerprint : A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPgFjGAAoJELymbjP2ci12MScIANRBXgWPSjeSZvSM2+bZmDrD
FefJfxMcCQFtTqWYePXIxFgqeeRcFc57+6alExBRU2g9nKZJEIPn2vj640XSTGlt
n5T4ItED1IVydzuEXFOAvgaTmZfKzTQkxQWp8QgL6pC7cYgnp2nL7gkbkedx1JyI
gCgCGomrHeWP2++vUS8UjtCccZdTJiOiWqoH2sBCKgEVcmFMgvYREK6hTou4DHU4
DcpureWuLvM4HWUlyfl37eXQOmy66A7uHGp3xF5sykQZ9aozaiMFjSoCqQOEVqO6
jHRnsQB6FfF/xU+F/UQK5PKZOm7e+KvrMjhIDfR+/DX3t0rkrtc69gg2O6duy4o=
=vdud
-----END PGP SIGNATURE-----
12 years, 2 months
/etc/rc.c/init.d/vdsmd set filters for libvirt at DEBUG level
by Peter Portante
Hello Folks,
I built vdsm from git and installed it on my system, and found that vdsm's init scripts are setting libvirt's logging levels to debug.
I was quite puzzled for a time trying to modify libvirt's config file to reduce the logging levels to warning, as it seemed that the vdsm install had placed proper settings in the libvirt config file itself.
Is there some history behind this log filter setting in the init scripts of vdsm that I should know about? Is the debug logging level of libvirt required for the correct operation of vdsm?
Thanks,
-peter
12 years, 2 months
Re: [vdsm] /etc/rc.c/init.d/vdsmd set filters for libvirt at DEBUG level
by Andrew Cathrow
Are we sure that performance concerns aren't manifested in 'real' environments? I think we should make some more effort to validate that assumption.
I agree with all the points below, but the worst part of the logging is the readability. Right now the log is fine for developers and support but from a user's point of view we may as well write it out in hex - it's very hard to decipher and makes self support for users very difficult.
We should default to INFO mode which should be understandable, and switching to debug should be simple and ideally possible from ovirt-engine.
Saggi Mizrahi <smizrahi(a)redhat.com> wrote:
of course you are right and just having everything log in debug is less then optimal.
We do only save 100 logs back so this will not fill up a decently sized hard drive.
The performance implication though existing don't really pop up in normal use.
It's true we usually don't need logs that far back but as I said it's really not an issue.
Even when you try and run a 1000 VMs you will not be blocked on logging trying to get written to disk I can assure you of that.
There are a lot of things we can do to improve the size footprint and readability of our logs.
The move to XZ instead of GZ was a simple way to save on space and there are plans on using an even more compressed representation for old logs.
We are also working on tools to help manage the log files.
You are correct though that setting the logging level back or changing the amount of logs kept shouldn't be such a bothersome process and you should open a bug on that for VDSM and it will be fixed.
----- Original Message -----
> From: "Peter Portante" <pportant(a)redhat.com>
> To: "Saggi Mizrahi" <smizrahi(a)redhat.com>
> Cc: vdsm-devel(a)lists.fedorahosted.org
> Sent: Thursday, April 5, 2012 3:57:08 PM
> Subject: Re: [vdsm] /etc/rc.c/init.d/vdsmd set filters for libvirt at DEBUG level
>
> Hi Saggi,
>
> ----- Original Message -----
> > From: "Saggi Mizrahi" <smizrahi(a)redhat.com>
> > To: "Peter Portante" <pportant(a)redhat.com>
> > Cc: vdsm-devel(a)lists.fedorahosted.org
> > Sent: Thursday, April 5, 2012 3:02:23 PM
> > Subject: Re: [vdsm] /etc/rc.c/init.d/vdsmd set filters for libvirt
> > at DEBUG level
> >
> > It's required for support purposes. When something goes wrong we
> > collect all the logs from the hosts, this way we can figure out
> > whats wrong without requiring someone to reproduce the problem with
> > the logging turned on.
> >
> > We are working on making the logs easier to filter when inspecting
> > them.
> > But the general idea is, if the information is in when can easily
> > get
> > it out, but doing it the other way around is impossible.
>
> While one can understand the pain of debugging complex systems,
> respectfully, this approach seems more problematic than helpful.
>
> First, it was filling up the system disk. In 10 minutes there were
> four compressed log files for libvirt alone, not to mention the
> vdsmd.log files. Talking with the rest of the performance team, we
> always turn all this off so that we don't loose our systems while
> testing.
>
> Additionally to get the logging to stop, one has to modify the vdsmd
> start up script to get libvirt to stop logging so much. Each time a
> modification was made to libvirt's configuration file to make it
> stop, libvirt kept up all its debug logging. All the documentation
> on the libvirt web page tells one what to do to affect its behavior,
> but in the presence of vdsm that is not the case. Somehow, that
> seems like a problem to have one subsystem completely override
> another with leaving any indication that it is doing so.
>
> Second, it is expensive to have such overhead. Compressing and
> maintaining arbitrary sized log files of text takes processing time
> away from the VMs. It was amazing to see how often xz would run on a
> box, not knowing it was related to maintaining these log files.
>
> There must be a better way than collecting all the data we could
> possibly need ahead of time just in case a problem comes up.
>
> Have you considered asserting the expected state of the system before
> embarking on changing that state?
>
> > In a nutshell order to filter out debug messages grep -v is you
> > friend.
>
> grep -v is not useful when your system disk fills up. :)
>
> And Why wouldn't an attacker use that fact in some sort of denial of
> service? And if the counter is to configure the log files so that
> they are processed more often and kept to a small number, then as
> the amount of data grows (like when multiple VMs are created) the
> original problem will get lost as it is truncated.
>
> So if we already a finite data set of sorts, why not drop using log
> files in favor of using a dedicated ring buffer that stores
> ultra-compressed binary data (enough to track the problem) with a
> tool that can format that ring buffer into useable output.
>
> When we were writing the thread library for Tru64 Unix and OpenVMS,
> such ring buffers were invaluable to help find complicated timing
> problems across multiple processors.
>
> Respectfully,
>
> -peter
>
> Ps - Blessed Passover to you!
>
> >
> > ----- Original Message -----
> > > From: "Peter Portante" <pportant(a)redhat.com>
> > > To: vdsm-devel(a)lists.fedorahosted.org
> > > Sent: Thursday, April 5, 2012 10:51:15 AM
> > > Subject: [vdsm] /etc/rc.c/init.d/vdsmd set filters for libvirt at
> > > DEBUG level
> > >
> > > Hello Folks,
> > >
> > > I built vdsm from git and installed it on my system, and found
> > > that
> > > vdsm's init scripts are setting libvirt's logging levels to
> > > debug.
> > >
> > > I was quite puzzled for a time trying to modify libvirt's config
> > > file
> > > to reduce the logging levels to warning, as it seemed that the
> > > vdsm
> > > install had placed proper settings in the libvirt config file
> > > itself.
> > >
> > > Is there some history behind this log filter setting in the init
> > > scripts of vdsm that I should know about? Is the debug logging
> > > level
> > > of libvirt required for the correct operation of vdsm?
> > >
> > > Thanks,
> > >
> > > -peter
> > > _______________________________________________
> > > vdsm-devel mailing list
> > > vdsm-devel(a)lists.fedorahosted.org
> > > https://fedorahosted.org/mailman/listinfo/vdsm-devel
> > >
> >
>
_______________________________________________
vdsm-devel mailing list
vdsm-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
12 years, 2 months
gerrit submmission, review, comment process questions
by Ryan Harper
Hi,
I'm just starting to work with gerrit and I've got a number of process
questions on how to map typical opensource project development with
gerrit.
In general, I'll work up a patch series, post to the mailing (including
a cover letter to describe the goal of the series and any details
around what testing was done), do review and discussion inline on the
list via email client. It's straight-forward to see all of the comments
about the whole series in an email thread. I'll respond to either
discuss issues or indicate I'll respin and include the fix.
Next, I go back and rework the code as needed, and post a v2 of the
series to the list. Repeat till code is accepted to completely rejected
in favor of another approach.
I'm trying to understand the gerrit analog of this. I've created my
patch series, used git-review to submit a topic branch. The web
dashboard shows each patch as it's own item for review. I get email
copies of my submissions with links to the gerrit link to the commit.
When folks review my patches I get emails with comments from them. In
some cases, when I look at the web ui, I can see that someone has
reviewed the patch, but the comments don't show up (even though they did
in the email; for example, http://gerrit.ovirt.org/#change,3287 shows
that Dan reviewed the patch and it has (1 inline comment). However, the
email I got in response to the review actually includes the comment.)
To respond to the comments of a single patch, I'm currently "reviewing"
my own patch, filling out the "cover" letter, but it's not clear to me
that everyone sees the cover letter before reading the patches etc.
Also, it's not clear to me if everyone can see the comments or only
folks who have been selected as reviewers. Is "reviewing" each one of
my own patches to reply to comments the preferred/best method within the
bounds of gerrit to exchange comments? Is it possible to reply to the
email that gets sent out with the comments directly and have gerrit
append those comments to the backend?
I'm now looking at the feedback and will be doing a v2; do I submit a
new topic branch or with git-review do the right thing if I resubmit
under the same topic branch ?
Thanks for your time.
--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
ryanh(a)us.ibm.com
12 years, 2 months
Re: [vdsm] [Engine-devel] VDSM REST API
by dfediuck@redhat.com
Hi Adam,
Just adding vdsm devel list, so vdsm folks
will be apart of this thread.
On 04/04/12 01:03, Adam Litke wrote:
> Hi all,
>
> At the oVirt Workshop in Beijing I learned about how the ovirt-shell dynamically
> discovers: the collections, resource schemas, and allowed actions based on an
> RSDL file and API xsd schema. I am working on a REST API for vdsm and would
> like to make my API compatible with the ovirt-engine api such that the same
> ovirt-shell program could work with either the engine api or the vdsm api.
>
> There are many differences between ovirt-engine and vdsm (namely that one is
> implemented in Java and the other in Python). I think the easiest way to test
> whether this is possible is to try and create a new, minimalist REST service with
> python Cherrypy. Such a service would have a root URL with no collections or
> actions. From my understanding I will need to write the following files:
>
> /api - A basic XML representation of the API root resource
> /api?schema - An xsd that describes the simple API
> /api?rsdl - An rsdl (XML file) that describes the available links
>
> for /api, I want to start with something dead-simple:
>
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <api>
> <messages>
> <message id='foo' href='bar'>
> Hello from vdsm!
> </message>
> </messages>
> </api>
>
> Once I can use ovirt-shell to list messages and show messages I will be happy to
> build on it. Can anyone help me figure out the minimal xsd and rsdl that would
> be needed for such an API to be consumable by ovirt-shell? Thanks for your
> help!
>
--
/d
Why doesn't DOS ever say "EXCELLENT command or filename!"
12 years, 2 months
nosetests for VDSM
by smizrahi@redhat.com
I submitted a series of patches the integrates python-nose with vdsm testing.
I've been getting a lot of questions in the form of "Why?" so I decided to answer them.
The vdsm team has a bunch of tests written that help make sure that VDSM is functioning properly.
Other Ovirt partners asked us to make the tests public and upstream and we were more then happy to oblige.
The tests internally used a custom made test infrastructure because the guy responsible for choosing the infrastructure said that there isn't a good open source alternative.
This is of course not the case and I've been planning to move to a more standard testing framework so I wouldn't have to maintain a costume one.
When Ayal asked if someone from our team could move all the tests upstream I volunteered to do it, mainly, because I thought it will be a good chance to finally change the test runner.
Our requirements are (In no particular order):
* Integrates with python unit tests
* Coverage support
* Test skipping (Some tests are only valid for some configurations)
* Configurable output
* Integrates with Jenkins
* Extensible using python
* Already packaged for RHEL\EPEL and Fedora
* Well supported and documented
nose answers all of these!
Also, an internal Red Hat survey shows that most teams here that work with python use nosetests and are very happy with it.
Further more, Open Stack also use nose and have done a lot of tool integration.
Seeing as we both use a similar toolset, we could share gerrit\Jenkins hooks.
Another thing I was also asked is why do the color wrapper around the output.
When I first got my hands around the original test runner we had here I wrote coloring to it.
For me, it's easier to skip a long list of tests and see the red pop out instead of looking for the word "FAIL".
12 years, 2 months
Re: [vdsm] [oVirt Jenkins] vdsm_check_code_violations - Build # 33 - Still Failing!
by abaron@redhat.com
----- Original Message -----
> From: "Jenkins oVirt Server" <jenkins(a)ovirt.org>
> To: abaron(a)redhat.com, vdsm-patches(a)lists.fedorahosted.org, danken(a)redhat.com, fsimonce(a)redhat.com
> Sent: Tuesday, March 20, 2012 6:38:17 PM
> Subject: [oVirt Jenkins] vdsm_check_code_violations - Build # 33 - Still Failing!
>
> vdsm_check_code_violations - Build # 33 - Still Failing:
>
> Check console output at
> http://jenkins.ovirt.org/job/vdsm_check_code_violations/33/console
> to view the results.
>
I reviewed the errors and the code, I couldn't find even one thing that is real (went through the first ~100 errors).
We need a conf file to filter these error types:
Instance of 'VmStatsThread' has no 'addStatsFunction' member
Passing unexpected keyword argument '__securityOverride' in function call
12 years, 2 months