On 12/01/2011 10:38 AM, Zane Bitter wrote:
Y'all will have noticed that I posted a patch a couple of days
ago with
a prototype of a Gofer agent for Matahari. As it stands, here is a rough
outline of what the patch does:
* Imports existing Gofer plugins and creates a QMF object (of class
org.matahariproject.gofer.Plugin) for each.
* Accepts RPC calls in the form of QMF calls to the "invoke" method of
the plugin object.
* Invokes RPC calls on the specified class and method in the plugin,
passing positional and keyword arguments which are supplied as a list
and map (respectively) with the parameter values in JSON format.
* Returns the result simply as a JSON string.
And what it does *not* do:
* Pass any out-of-band data with the RPC call - e.g. maintenance
windows, class constructors, shared secrets. In principle, all of these
could be added as parameters to the QMF call
* Distinguish between normal return values and exceptions - it is
currently up to the client to deduce this from the JSON string.
* Provide any way for a client to call methods RPC-style (i.e. by just
calling them, rather than going through the QMF API).
For a number of reasons I have put this functionality in a separate
agent rather than adding it to, say, the host or sysconfig agents. In
general we want to avoid a proliferation of agents, but the following
are valid reasons for creating a separate agent instead of adding
functionality to an existing one:
* The SELinux permissions required can be narrowed by putting the
functionality in a separate agent.
* The user could conceivably want the functionality executed with a
different user id.
* The user could conceivably want to enable or disable the functionality
independently of the functionality of existing agents.
All but the first apply to the Gofer agent.
Now that I am somewhat familiar with the internals of how Gofer works,
and thanks to the discussion with Jeff that has been happening here on
the mailing list, it seems that there are essentially two paths we could
go down to fulfil the requirements.
To recap: the requirement is that we provide a way for sysadmins/devops
to easily write custom APIs in a high-level language that integrates
with their existing code (i.e. Ruby& Python), and expose them over
Matahari using the same infrastructure as other Matahari agents. This
means only one lot of brokers, configuration, certificates,&c. to deal
with.
The big unknown is around whether we need to provide to clients an
RPC-style API (i.e. where calling remote code looks not much different
to calling local code) as well as a native Matahari API. This is
fundamentally what Gofer is about (in Python) and also what MCollective
does (in Ruby -
http://docs.puppetlabs.com/mcollective/simplerpc/). On
the other hand, I'm not sure if this is really needed, or if it even
makes sense to talk about it in an environment where the client can be
written in one of several languages (Gofer supports both Python and Ruby
client interfaces to the Python plugins).
Option 1: Continue with the Gofer port
--------------------------------------
Essentially, attempt to get all of Gofer working over Matahari. Existing
Gofer plugins would continue to work; existing clients would need only
to switch over to the new backend (which should be as simple as "from
matahari import gofer" at the top of the file.
Work required:
* Parse the method output JSON and either return ret_val or raise a QMF
exception with the exception data.
* Implement Gofer-specific code in the Matahari Python API to simplify
the interface.
* Implement a client-side library to dispatch RMI calls to the
QMF/Matahari backend. (Needs to be done for both Python and Ruby.)
* Add authorisation parameter to invoke method.
* Add support for delayed/recurring actions.
* Add support for class constructors.
Advantages:
* Compatibility with existing Gofer plugins and clients.
* Reuses existing Gofer configuration&c. (already defined location for
plugins, config files, config format...)
Disadvantages:
* QMF API becomes complex
* We may have to do a lot of work to support features we are not
necessarily that concerned about.
Option 2: Start from scratch
----------------------------
Forget about Gofer and implement the simplest possible agent that will
allow us to call out to Python (and later Ruby) code. Basically what
Jeff suggested in his last email.
Work required:
* Define a plugin format, location, settings, SELinux config&c.
* Implement plugin loading and method call dispatching.
* Implement Gofer-specific code in the Matahari Python API to simplify
the interface.
Advantages:
* We only have to support the features we need and that make sense with
a QMF backend.
* Possibly a simpler interface to also write a Ruby agent against?
Disadvantages:
* No RPC API initially
* If we were to end up implementing a lot of Gofer's features, we would
have reinvented the wheel.
* No support for existing(/future) Gofer plugins/clients
Before writing this email I had no real clue which path to pursue. After
writing it, I am leaning toward option 2. Here are the issues I think we
need to consider:
Questions:
* Do we need an RPC API?
I think this is very important. If we're going to go through so much
effort to make it easy to write the agent side, we should put the same
effort into the client side.
You mentioned MCollective. I think it should definitely be used as
inspiration here. They make both the client and agent side easy.
http://docs.puppetlabs.com/mcollective/simplerpc/
* How important is it that we maintain compatibility with existing
Gofer
plugins?
Perhaps the bigger issue here is whether or not this ends up resulting
in porting all of Gofer to Matahari infrastructure. IIRC, this whole
thing started because there seemed to be at least some overlap between
the goals of Matahari and Gofer. Gofer also offered something that we
were lacking with Matahari: an easier interface for writing agents and
clients in a higher level language.
In an ideal world, we could combine efforts and merge the projects. If
that's not going to be possible or just not practical, then I don't
think compatibility matters that much. If we do come out of this
merging efforts, then compatibility would seem important.
* Do we need method-level authorisation using shared secrets, or
could
we effect authorisation by other means? I believe we should have some
kind of authorisation at at least the per-plugin level. (What is our
plan for authorisation with existing Matahari agents today?)
Well ... we do need it. I don't think it's a problem that has to be
solved as a part of this Gofer work, though. It's a very big problem
that applies to all of QMF. We need to come up with a plan to solve it
at that level. Once we figure that out, we can look at how to leverage
it in these APIs.
* Are delayed/repeated actions important, or should we leave this to
the
client?
Is this something supported in Gofer today? Do you have any pointers to
what it looks like?
* Do we need to (eventually) support Ruby plugins?
Eventually, yes. I've talked to at least one potential consumer of this
work that wants to do everything in Ruby (as that is what their existing
code does).
This concludes the most epic email in history. Thanks for reading.
It was so epic that I just couldn't trim any of it! Thanks for writing.
I'm actually leaning toward the first option but that's heavily
dependent on what both you (Zane) and Jeff think about the possibility
of merging projects. Is that possible? Desirable?
--
Russell Bryant