----- Original Message -----
From: "Dan Kenigsberg" danken@redhat.com To: "Ayal Baron" abaron@redhat.com Cc: "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Tuesday, 14 February, 2012 10:40:40 AM Subject: Re: [vdsm] flowID schema
On Mon, Feb 13, 2012 at 12:06:46PM -0500, Ayal Baron wrote:
----- Original Message -----
On 02/13/2012 02:28 PM, Ayal Baron wrote:
...
is that it (ab)uses an http header for carrying FlowID,
Yes, it certainly does appear to overload it. I would be nice to have something formal given to it by engine, but I can appreciate the difficulty implementing such a scheme.
Technically I disagree, this is a cross cutting concern which has nothing to do with any specific call hence it should be passed as a header, that is actually rather elegant.
To the specific matter at hand though. what would really be nice is solving the real problem properly, and not contaminating the API and the log with things which have marginal benefit if at all.
going back to the 'grep' issue. vdsm logs are verbose. they are multi-threaded as well. I think this should be more than just about finding the entry point of the flow, then identifying for this specific log format how to trace it, which would require writing a log analyzer with plugins for each component. having all lines which are relevant to a flow with a flowid logged in them would make it much easier to get all (or most) of relevant parts of the flow (most, since something orthogonal to the flow may have happened affecting it, like loss of network)
I'm sorry but what you're proposing is to make the log even more difficult to read for absolutely NO reason. I haven't seen 1 good reason to add more to the log. What we should be focusing on is:
- adding the relevant data that is needed to the engine log so
that most of the time users wouldn't need to go the host 2. reducing the verbosity of the vdsm log and increasing readability (the flow ID does exactly the opposite).
As opposed to most people here who are thinking that this sounds like a good idea, I actually have debugged at least dozens of issues in engine and vdsm and can assure you that not once would this have been beneficial to me.
When I debug an Engine-related issue, I tend to find a silly API call in Vdsm. Then I have to start correlating this to Engine logs. This step can be made quicker and less error-prone by logging FlowID both Engine and Vdsm. To me, this is the 1 good reason for logging FlowID on API entry in Vdsm.
What I've been saying all along. But Ayal will just say he didn't see a good reason brought up :)
However, I find adding FlowID to each and every log line a bit excessive. Our log is too cluttered as it is. Logging FlowID whenever a new thread is spawned makes more sense to me.
In that case, this needs to be done in a way that is easily grep-able and clearly visible when skimming over a logfile, in a bird's eye view, sort of speak. So far, looking for the "run and protect" line for a specific thread, making sure it pertains to the same thing the engine kicked off in it's own log, and then following that thread through was never very easy. Manageable with some experience, but not easy.
What was mostly missing in the engine logs was understanding what thread in engine called what operation in vdsm and what vdsm's response was. In 3.0 my understanding is that engine fixed this so this entire feature will be counter productive (will make logs less readable and harder to decipher, adds complexity to the API and adds complexity to users of the rest API). All cross hosts issues stem from *different* flows, so this would not help in this case and single host issues are easily traceable today (and you *never* need to follow an entire flow, it's entirely redundant and inefficient).
I'm more than willing to show this on any set of logs by the way and would be happy to be proven wrong.
More often than not by the way, the issue is that inside a specific call (i.e. 1 verb, not a flow) people are not proficient in finding the offending line (which is why I wrote the 'how to read the vdsm log' wiki).
vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel