On Mon, Jun 18, 2012 at 03:14:15PM -0500, Anthony Liguori wrote:
On 06/18/2012 10:02 AM, Saggi Mizrahi wrote:
I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine.
Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks.
We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future.
In order to achieve this though we need to do several things:
- Declare API supportability guidelines
Adding danpb and DV as I think they can provide good advice here.
Practically speaking, I think the most important thing to do is clearly declare what's supported and not supported in more detail than you probably want to. Realistically, you have to just support whatever you have. I don't know that designing a "supportable" interface can be really successful unless you start with that tomorrow.
So basically, unless you plan on removing the XML-RPC interface in the next release, you should plan on supporting it forever...
Agreed, whether you like it or not, whatever you ship becomes the supportable interface. If there are warts you don't like then at least document your deprecation procedure. If deprecating something you /must/ provide apps an alternative viable API for the same functionality, or you'll find they just continue using the deprecated stuff & you'll never be able to get rid of it. Also when deprecating, you need to leave a couple of releases where both old & new APIs exist, to give apps time to adjust.
- Decide on an API transport (e.g. REST, ZMQ, AMQP)
We spent so much time trying to find the best transport in QEMU with the resulting being something I'm ultimately unhappy with.
The best decision we've made recently on this front is to move to a schema-based RPC mechanism where the transport code is all autogenerated. Python has an advantage in that it supports introspection although a disadvantage in that it's easy to end up with an ad-hoc interface by relying on passing around dictionaries.
My advice here would be to pick a primary/native transport that
- Easily consumed by the widest possible set of platforms - Has minimal (near zero) infrastructure requirements - Is easily bindable/translatable to other API transports
Once the primary transport is decided, declare that any other transports are to be added as external agents which bridge from one model to the other. Don't try to implement multiple primary transports internally, or you'll fall into the trap where each transports has different subsets of your functionality exposed.
If you do pick a message bus based transport (ZMQ/AMQP) as the primary one, then you definitely want a REST or XML-RPC bridge to that to make it more easily consumable
- Make the API easily consumable (e.g. proper docs, example code, extending the API, etc)
Documentation is by far the most important thing IMHO. I actually think that simply taking the existing XML-RPC interface and adding documentation ought to be the first step even..
Yep, good docs are important. libvirt kinda sucks in this regard, (but somehow we've suceeded despite this flaw).
- Implement the API itself
I think the biggest risk in an effort like this is letting perfect become the enemy of good. If the goal is to open VDSM up to other applications, you can start today but just documenting what you have with plans to deprecate and improve later.
Agreed, from my experiance both with libvirt & QMP I'd say don't worry too much about getting APIs wrong. Expect that you'll make mistakes and you'll need to add new APIs later to correct these. I don't mean accept any old API that is proposed - certainly do critical analysis of all new APIs, but don't hold up APIs indefinitely because they are not quite perfect. This really hurt us with QMP for a period of time.
Honestly, worrying about XML-RPC vs. REST vs. AMQP is likely going to result in a lot of bike shedding and grand plans.
Pick REST ;-P
Daniel