----- Original Message -----
From: "John Sanda" <jsanda(a)redhat.com>
To: rhq-devel(a)lists.fedorahosted.org
Sent: Tuesday, June 5, 2012 6:05:53 PM
Subject: Re: Modular scripting in CLI
Most languages provide some some mechanisms for code reuse, be it at
the
level of a function, classs, package, shared library, etc. JavaScript
is
not on of those languages. This is a major shortcoming. This is not
an
easy problem to solve, and fortunately as Lukas has pointed out we
don't
have to. There are already several existing solutions. I am in favor
of
moving to Rhino 1.7R3. In addition to CommonJS module support,
JavaScript objects implement Java collection interfaces (finally) in
this release.
As for implementing our own script engine, I wouldn't think that it
would be too much effort especially since Rhino already implements
all
of the javax.script APIs. This brings up a related point though that
ties in with my email about doing away with built-in functions. The
primary advantage of working through the javax.script APIs is that it
provides a common set of APIs across script engines and languages.
This
can be really nice for interopability when dealing with polygot
programming. We however, only support JavaScript so I think it is
worth
asking what benefit is there of going through the extra layers and
complexity of the javax.script APIs versus directly using Rhino.
I think the main reason for us to go through javax.script APIs is not to close the doors
from supporting other languages, too. I didn't clearly stress it before but I do
believe that we should leave it for our community to decide what scripting language
they'd prefer by simple "voting with action". We have our rhq-samples github
project which I think would be a
great place for people to share utility scripts in whatever scripting language. By letting
the community build around our support for multiple langs we effectively broaden the
choice for the community and also provide another area for external contributions. This
largely overlaps with the REST "problem" you mention below.
Two
of
the big features of the CLI shell are tab completion and command
history. The Rhino REPL (like just about every other REPL for other
languages I have used) provide those features.
It seems to me that moving to using Rhino directly is something we
should strongly consider. The difficult problem of module support is
already solved and we have less code to maintain.
There's a lot to like about this idea, but given the point I raised above about
sticking to javax.script and also because of the fact that Rhino's REPL has some
feature disparity with our existing CLI (in terms of the detailed code completion we
provide), I'm not sure it would be easy to try and (re)use Rhino's REPL. CommonJS
support as well as enhancing the scope with RHQ's contextual variables shouldn't
be too difficult but modifying the code completion as well as adding the interactive
editing features we (somewhat) provide could prove complex.
I like the idea of being able to share code between client-side and
server-side alert CLI scripts. I think that it something worth
further
investigation.
Me too :)
Along those lines, I think we should consider
including a
library like underscore.js. JavaScript provides a very minimal
standard
library and libraries like underscore.js help fill in a lot of those
gaps. I've used it in CLI scripts before to avoid rewriting common
utility functions.
One of the advantages of tying ourselves to CommonJS is that it defines a number of module
APIs to form a sort of standard javascript library. For example Narwhal implements quite a
large number of them (
https://github.com/tlrobinson/narwhal/tree/master/lib).
With respect to additional language support, I think that ought to
be
part of a related discussion on the CLI, the REST APIs, and how they
co-exist going forward as the REST APIs mature.
Yes. With REST our support for CLI becomes nearly pointless. The only reason I think it is
worth sticking with our current remote APIs (and to a lesser extent with the CLI as it is
or rather will be ;) ) is that the remote APIs aren't exclusively used only by CLI but
rather are just a public "endpoint" that can be used by any Java-based client
(and it is being used, as you know). I'll wander slightly off topic now, but I do
believe that if nothing else then we must at least close the gap between remote API and
REST (as far as I know they have truly diverged now with REST no longer offering only a
subset of what's available in remote API).
Another reason for sticking with remote API in the CLI is the fact that we can run scripts
on the serverside - currently only as alert scripts but I do hope for some other
scriptable serverside functionality in the future. Using REST API directly on the server
seems like a little bit of an overkill to me (and also remember the problems we had
sharing the session between our GUI app and the REST war - this might or might not apply,
I think there would be some differences to how reporting tried to use REST).
On 06/04/2012 11:59 AM, Lukas Krejci wrote:
> Hi all,
>
> this is to describe my recent work towards the support of modular
> scripts in our CLI. The code I am talking about here is located in
> a separate branch in our repo
>
http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=shortlog;h=refs/heads/lk....
>
> First of all to quickly describe what script modularity means - it
> is the ability of a script to somehow include the code from
> another script and use it. I.e. it is a way of splitting up your
> scripts into reusable parts.
>
> I've done some preliminary investigation of possible approaches a
> couple of months ago, captured in
>
http://wiki.rhq-project.org/display/RHQ/Script+Modularity. My main
> point during that investigation was that we should strive for
> bringing the support for CommonJS modules into RHQ CLI. In that
> investigation, the only alternative I found was
http://ringojs.org
> - but that is an awful overkill, because it is a full runtime for
> running web applications written in javascript (on the
> serverside).
>
> After that, John Sanda pointed out to me that Rhino 1.7R3 itself
> (the javascript interpreter for java) contains a CommonJS
> compliant module loader. That means that we actually would not
> need nothing more than Rhino 1.7R3 for gaining module support in
> RHQ CLI. Unfortunately that is easier said than done. The version
> of Rhino that comes bundled with the JRE is not easily swapped for
> different version - I guess that is for a good reason. There is
> therefore no (not awfully hacky) way of using Rhino 1.7R3 with the
> default javascript script engine that comes with JRE.
>
> In my opinion having our own script engine for a specific version
> of Rhino is by far the easiest approach for gaining the module
> support however you slice it. The reason for this is that while
> module loaders written in pure javascript are possible (requirejs
> on Rhino), the implementation looks several orders of magnitude
> more complex than what is done in Java (at least to an untrained
> eye of a Java guy like me). The other reason is that I found no
> pure javascript-based CommonJS-compliant module loader (requirejs
> uses a slightly different approach called AMD) and given the
> complexity of requirejs code, I'd really hate to write our own.
>
> The disadvantage of having our own script engine is obviously that
> we then become "owners" of that script engine and have to maintain
> it. In my humble opinion that is not a big deal actually because
> thanks to the licensing of the Phobos project
>
https://svn.java.net/svn/scripting~svn/, we can bundle their
> script engine (which is almost identical to the one bundled with
> JRE, the same people wrote it actually) and provide our own little
> additions/modifications to it. Unfortunately we cannot just extend
> their existing impl and use it as a library but because of the way
> the code is written and what we need to do, we have to modify the
> source code of the script engine - in fact only a single file
> needed update to get what we want.
>
> So in another words we now have a script engine
>
(
http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=modules/enterpris...)
> that is equal (as far as I can tell) to the one bundled with JRE
> (including the security features) and in addition makes the
> "require" function and CommonJS support as such available out of
> the box.
>
> Another very important aspect of Rhino's CommonJS implementation is
> that the "module providers", i.e. the things that are responsible
> for locating the source code of the module when it is being
> loaded, are pluggable. This enables us to provide an
> implementation that is not only able to locate a module somewhere
> on a filesystem, but also for example in RHQ's repository. Think
> about this code snippet:
>
> var my_stuff = require('repo://my-rhq-repository/my_stuff.js');
> my_stuff.reign_the_world();
>
> Sharing common modules between remote CLI scripts and server-side
> alert notifications? Putting all the utility code in a public
> repository on the server and not having to download them to each
> CLI installation? Prepackaging RHQ samples and/or utility code on
> the serverside so that they can be used by alert scripts, too?
> (I'd imagine this being handled by yet another "scheme" like
> "require('rhq://util');" being equivalent to today's
"exec -f
> 'samples/util.js';" with the exception that the actual code would
> be stored on the serverside so that both CLI scripts and alert
> scripts could use it).
>
> But that's not all that has been done. You may remember from the
> last year's F2F meeting that I put together a crude version of
> python-backed CLI just during John Sanda's presentation on another
> topic (and I apologize to John once again for being so rude - it
> WAS an interesting presentation ;) ). I was able to do that that
> quickly because our current code is 99% language independent. I
> thought that if we are going the way of custom script engines and
> bundling language interpreters it would be really a shame not to
> add that 1% and make the CLI code completely language independent
> and push the language support into standalone "modules". Why do I
> think it is useful? Well, speaking of John Sanda again, he likes
> Clojure, admins, I imagine, would prefer python over javascript,
> script kiddies would probably prefer ruby over python and someone
> else might like groovy. Do we need to support them all in JBoss
> ON? I'd say no, but at the same time we should not artificially
> limit our users - be it RHQ or JBoss ON users - from using the
> scripting language of their choice if the work required on our
> side amounts to adding 1% to the 99% complete puzzle.
>
> So that's almost done, too... There are a couple of new modules:
> modules/enterprise/scripting/api defining the API that needs to be
> implemented by new language support modules and
> modules/enterprise/scripting/javascript and even
> modules/enterprise/scripting/python (which is not complete yet).
> The language support modules are discovered using
> META-INF/services mechanism and are made available for
> instantiation through RHQ's ScriptEngineFactory (which required no
> API changes for this, remember we were 99% there). The thing that
> is missing is the updates needed to our build process for bundling
> the javascript language support with the CLI (maybe we could later
> think about having other language support modules being
> downloadable from the server from Administration/Downloads?) and
> porting the code completion code from the CLI itself into the
> javascript language module (which should be fairly easy as far as
> I checked). (Other languages would probably lack the code
> completion altogether but I think we should think about providing
> some interactive API help rather than trying to produce a perfect
> REPL with code completion - and that could work across different
> languages more easily.)
>
> So there you go. Bringing modularity support to javascript wasn't
> that hard after all and as a bonus we might gain the ability to
> support other scripting languages, too ;) Unfortunately there is
> nothing to try out yet (you can only stare at the code), because
> as I wrote above, I didn't connect all the dots yet (even though
> I'm close). I wanted to give you this overview of what's been done
> and what it'll bring us so that we can discuss the approach I took
> and hopefully make some enhancements to it.
>
> Cheers,
>
> Lukas
>
>
>
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel(a)lists.fedorahosted.org
>
https://fedorahosted.org/mailman/listinfo/rhq-devel
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-devel