On Tue, Jun 26, 2012 at 05:37:26PM +0300, Dan Kenigsberg wrote:
On Mon, Jun 25, 2012 at 04:19:28PM -0500, Adam Litke wrote:
> On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote:
> > On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote:
> > > On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote:
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Ryan Harper" <ryanh(a)us.ibm.com>
> > > > > To: "Adam Litke" <agl(a)us.ibm.com>
> > > > > Cc: "Anthony Liguori" <aliguori(a)redhat.com>,
"VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org>
> > > > > Sent: Friday, June 22, 2012 12:45:42 PM
> > > > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt
host manager
> > > > >
> > > > > * Adam Litke <agl(a)us.ibm.com> [2012-06-22 11:35]:
> > > > > > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote:
> > > > > > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >----- Original Message -----
> > > > > > > >>From: "Deepak C Shetty"
<deepakcs(a)linux.vnet.ibm.com>
> > > > > > > >>To: "Ryan Harper"
<ryanh(a)us.ibm.com>
> > > > > > > >>Cc: "Saggi Mizrahi"
<smizrahi(a)redhat.com>, "Anthony Liguori"
> > > > > > > >><aliguori(a)redhat.com>, "VDSM
Project Development"
> > > > > > > >><vdsm-devel(a)lists.fedorahosted.org>
> > > > > > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM
> > > > > > > >>Subject: Re: [vdsm] [virt-node] VDSM as a
general purpose virt
> > > > > > > >>host manager
> > > > > > > >>
> > > > > > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote:
> > > > > > > >>>* Saggi Mizrahi<smizrahi(a)redhat.com>
[2012-06-18 10:05]:
> > > > > > > >>>>I would like to put on to the table
for descussion the
> > > > > > > >>>>growing
> > > > > > > >>>>need for a way
> > > > > > > >>>>to more easily reuse of the
functionality of VDSM in order to
> > > > > > > >>>>service projects
> > > > > > > >>>>other than Ovirt-Engine.
> > > > > > > >>>>
> > > > > > > >>>>Originally VDSM was created as a
proprietary agent for the
> > > > > > > >>>>sole
> > > > > > > >>>>purpose of
> > > > > > > >>>>serving the then proprietary version
of what is known as
> > > > > > > >>>>ovirt-engine. Red Hat,
> > > > > > > >>>>after acquiring the technology,
pressed on with it's
> > > > > > > >>>>commitment to
> > > > > > > >>>>open source
> > > > > > > >>>>ideals and released the code. But just
releasing code into
> > > > > > > >>>>the
> > > > > > > >>>>wild doesn't
> > > > > > > >>>>build a community or makes a project
successful. Further more
> > > > > > > >>>>when
> > > > > > > >>>>building
> > > > > > > >>>>open source software you should aspire
to build reusable
> > > > > > > >>>>components instead of
> > > > > > > >>>>monolithic stacks.
> > > > > > > >>>>
> > > > > > > >>>Saggi,
> > > > > > > >>>
> > > > > > > >>>Thanks for sending this out. I've
been trying to pull
> > > > > > > >>>together
> > > > > > > >>>some
> > > > > > > >>>thoughts on what else is needed for vdsm
as a community. I
> > > > > > > >>>know
> > > > > > > >>>that
> > > > > > > >>>for some time downstream has been the
driving force for all of
> > > > > > > >>>the
> > > > > > > >>>work
> > > > > > > >>>and now with a community there are
challenges in finding our
> > > > > > > >>>own
> > > > > > > >>>way.
> > > > > > > >>>
> > > > > > > >>>While we certainly don't want to make
downstream efforts
> > > > > > > >>>harder, I
> > > > > > > >>>think
> > > > > > > >>>we need to develop and support our own
vision for what vdsm
> > > > > > > >>>can be
> > > > > > > >>>come,
> > > > > > > >>>some what independent of downstream and
other exploiters.
> > > > > > > >>>
> > > > > > > >>>Revisiting the API is definitely a much
needed endeavor and I
> > > > > > > >>>think
> > > > > > > >>>adding some use-cases or sample
applications would be useful
> > > > > > > >>>in
> > > > > > > >>>demonstrating whether or not we're
evolving the API into
> > > > > > > >>>something
> > > > > > > >>>easier to use for applications beyond
engine.
> > > > > > > >>>
> > > > > > > >>>>We would like to expose a stable,
documented, well supported
> > > > > > > >>>>API.
> > > > > > > >>>>This gives
> > > > > > > >>>>us a chance to rethink the VDSM API
from the ground up. There
> > > > > > > >>>>is
> > > > > > > >>>>already work
> > > >
> > > > > > > >>>>in progress of making the internal
logic of VDSM separate
> > > > > > > >>>>enough
> > > > > > > >>>>from the API
> > > > > > > >>>>layer so we could continue feature
development and bug fixing
> > > > > > > >>>>while designing
> > > > > > > >>>>the API of the future.
> > > > > > > >>>>
> > > > > > > >>>>In order to achieve this though we
need to do several things:
> > > > > > > >>>> 1. Declare API supportability
guidelines
> > > > > > > >>>> 2. Decide on an API transport
(e.g. REST, ZMQ, AMQP)
> > > > > > > >>>> 3. Make the API easily consumable
(e.g. proper docs,
> > > > > > > >>>> example
> > > > > > > >>>> code, extending
> > > > > > > >>>> the API, etc)
> > > > > > > >>>> 4. Implement the API itself
> > > >
> > > > In the earlier we'd discussed working to have similarities in the
modeling between the oVirt API and VDSM but that seems to have dropped off the radar.
> > >
> > > Yes, the current REST API has attempted to be compatible with the current
> > > ovirt-engine API. Going forward, I am not sure how easy this will be to
> > > maintain given than engine is built on Java and vdsm is built on Python.
> >
> > Could you elaborate why the language difference is an issue? Isn't this
> > what APIs are supposed to solve?
>
> The main language issue is that ovirt-engine has built their API using a set of
> Java-specific frameworks (JAXB and its dependents). It's true, if you google
> for 'python jaxb' you will find some sourceforge projects that attempt to
bring
> the jaxb interface to python but I don't think that's the right approach.
If
> you're writing a java project, do things the java way. If you're writing a
> python project, do them the python way. Right now I am focused on defining the
> current API (API.py/xmlrpc) mechanically (creating a schema and API
> documentation). XSD is not the correct language for that task (which is why I
> forsee a divergence at least at first). I want to take a stab at defining the
> API in a beneficial, long-term manner.
>
> 1) Completely define the current XMLRPC API including all functions, parameters,
> and return values. Complex data structures can be broken down into their basic
> types. These are: int, str, bool, list, dict, typed-dict, enum. I have already
> started this process and am using Qemu's QAPI schema language. You can see
that
> here [1]. For an example of what that looks like describing the vdsm API see
> this snippet [2].
>
> 2) Import the parser/generator code from qemu for the above schema. Vdsm will
> require a few extensions such as typed-dictionaries, tuples, and type aliases.
> Adapt the generator so that it can produce a libvdsm which provides API language
> bindings for python, c, and java.
>
> 3) Implement a vdsm shell in terms of libvdsm. In fact, this can be largely
> auto-generated from the schema and accompanying documentation. This can serve
> to model how new transports can be written. For example, an AMQP implementation
> can be implemented entirely outside of the vdsm project if we wished. It only
> needs to talk to vdsm via libvdsm.
>
> Easy as 1,2,3 :)
>
> [1]
http://git.qemu.org/?p=qemu.git;a=blob;f=qapi-schema.json;h=3b6e3468b440b...
> [2]
http://fpaste.org/rt96/
>
> Probably more than you bargained for when asking for more info... :)
Indeed!
I am still at a loss why the languages take such a prominent place in
your choice for an API. A good API is easily consumable by any language.
The language bindings come first here because they can be easily generated.
They are a natural next step once you have libvdsm (which is basically a
generated API.py).
Your documentation of the current API, as shown in
http://fpaste.org/rt96/
is impressive and useful, but I'm not sure I like where this is going.
Thanks. My rationale is to start from where we are today and fully catalogue
what we have today. We are never going to be able to cut engine over to a
brand-new API in an instant so we need to support it for some time. If we
design an API that is changeable in a backwards-compatible way, then we can
start improving it as we go forward. What I want to avoid is having two API
entrypoints: one for engine and one for everyone else.
The XML-RPC API served us well when it was an internal API, which
had
one server, one client, with teams sitting on the other side of the
corridor. Petrifying this API as-is, for multiple clients in multiple
use cases, seems risky to me.
Once you have libvdsm, that it becomes the interface. The fact that we will
start out by using xmlrpc is really just an internal implementation detail.
Actually, xmlrpc is probably fine for an internal remote protocol between
libvdsm and vdsmd. If we want to do AMQP or something down the line, we'd just
write against the libvdsm interface.
We know there are bad things in the current API, and we have an
opportunity to fix them before our ecosystem becomes too complex. REST++
Yes, it's far from perfect right now but it never will be. One step I'd like to
take is to get the API fully specified in a schema doc and then have a round of
initial cleanups before we bake it. Maybe we can reduce the number of psuedo
boolean types (True/False, 'true'/'false', 'on'/'off',
etc.) and eliminate
string representations of integer and float types from the API. This would be
the time to quickly make other changes of this nature. The nice thing about a
formal schema is that it makes it clear where the API is ugly. Then we can bake
the initial schema and generate libvdsm. From that point, future API extension
and modification would take place using the compatibility rules.
--
Adam Litke <agl(a)us.ibm.com>
IBM Linux Technology Center