Hi,
Since Rails makes it appear trivial, I hoped we could make some quick
wins wrt to providing a more RESTful API for conductor in the coming
release.
It turns out that we have a fair bit of work to do here, so I've just
summarized my thoughts below.
Does a redmine issue tracking this currently exist? I'd be happy to link
to these notes and perhaps split it up into more discreet redmine tasks.
Cheers,
Mark.
= The API =
An XML/JSON over HTTP interface is not automatically a RESTful
interface. There's a fairly academic definition of REST, but what I look
for is:
- Resource based - i.e. collections of resources with
GET/PUT/POST/DELETE methods
- Resource identifiers - i.e. all resources have an ID and can be
referenced by a URL
- Sane document format representing the resources - i.e. reasonable
looking XML/JSON documents
- GET is a safe method and PUT is idempotent - i.e. GET doesn't
result in state changes
- A nod to HATEOAS - i.e. documents contain links to other resources,
no major reliance on query parameters
- An entry point - i.e. GET on the base URL lists the available
resources
- HTTP authentication supported - i.e. no need for cookies
Another indicator of whether your RESTful API is sane is how well it
works existing tools. For example, with a simple tool like resty, you
should be able to do:
$> curl -L
http://github.com/micha/resty/raw/master/resty > resty
$> . ./resty
$> resty
http://localhost:3000/conductor -u admin:password -H 'Accept:
application/json' -H 'Content-type: application/json'
$> GET /
[ {rel => "pools", href => "/conductor/pools"}, {rel =>
"deployments", href => "/conductor/deployments"}, ... ]
$> GET /pools
[ {"name" => "default_pool", "id" => 1,
"url" => "/conductor/pools/1", ...}, ...}]
$> POST /deployments --data-binary '{"name" => "test",
"deployable_url" => "...", "realm" => {"id"
=> "1"}}'
{"name" => "test", id => "2", href =>
"/conductor/deployments/2", "deployable_url" => "...",
"deployable_xml" => "...", "realm" => {"id"
=> "1", "href" => "/conductor/realms/1"}, ...}
Good examples of API styles which we should try to follow include the
Deltacloud and RHEV-M APIs. See also:
http://fedoraproject.org/wiki/Cloud_APIs_REST_Style_Guide.
= Current State =
The providers, provider_types and provider_accounts collections have
some XML representation support which aeolus-image uses.
We also have minimal JSON support in the pools, instances and
deployments collections.
You can create various resources using POST and query parameters even
where JSON or XML isn't supported e.g.
$> POST /pools -d pool[name]=foo -d pool[pool_family_id]=1
Finally, we already have HTTP authentication support allowing you do
just do:
$> resty
http://localhost:3000/conductor -u admin:password
= Default Media Type =
If no media type is specified via an Accept header or otherwise, we
should have a predictable default media type.
Preferably, that would be XML or JSON, since browsers always specify a
media type.
Rails 2 defaults to the first format in the respond_to block, which is
often either JS or HTML in our code.
It looks like Rails 3 always default to HTML[1], so either we change
that to XML/JSON or stick with HTML by default.
[1] - See ActionDispatch::Http::MimeNegotiation::formats()
= XML Document Format =
Our basic XML document format should be e.g.
<bars>
<bar id="..." href="/conductor/...">
<name>...</name>
<property>value</property>
...
<bar>
</bars>
More specific modelling would help in cases where there are multiple
related properties e.g.
<user id="..." href="/users/...">
<name>foo</name>
<logins>
<count>2</count>
<failed_count>0</failed_count>
<current_login>
<ip>...</ip>
<date>...</date>
</current_login>
<last_login>
<ip>...</ip>
<date>...</date>
</last_login>
</logins>
</user>
Also, if you consider the provider_type_id property in providers, do we
want to automate it's serialization as:
<provider id="..." href="/providers/...">
<name>foo</name>
<provider_type id="1" href="/providers_types/1"/>
...
</provider>
Or prefer simply:
<provider id="..." href="/providers/...">
<name>foo</name>
<type>ec2</type>
...
</provider>
Automated serialization of the ActiveRecord models only gets us so far,
we need to do XML-specific modelling too.
= JSON Document Format =
Our JSON document format should be a very simple mapping from the XML
format. That should be easier to implement and document. It also doesn't
make sense to have a different serialization.
For example:
[ {"id" => "...", "href" =>
"/conductor/...", "name" => "...",
"property" => "value", ... }, ... ]
Note that we currently do:
ActiveRecord::Base.include_root_in_json = true
which gives:
[ {"bar" => {"id" => "...", "href" =>
"/conductor/...", "name" =>
"...", "property" => "value", ... } }, {"bar"
=> {...} }, ... ]
This seems fairly unnatural and redundant, so we should probably turn it
off. However, at least the code in backbone.rails.js which relies on
this format would need to be updated.
JSON serialization happens using the to_json and as_json methods. The
latter returns a string and the former returns something which is
automatically serializable.
For example, to omit some internal properties from the JSON, you can do:
class Provider < ActiveRecord::Base
...
def as_json(options = nil)
options[:except] ||= []
options[:except] += [:created_at, :updated_at]
options[:except] += [:lock_version]
super(options)
end
...
end
In Rails 3, there is a serializable_hash method which is also used for
automatic XML serialization. This is probably what we want to override.
= Links and Entry Point =
The HATEAOS constraint basically means that all parts of the API should
be accessible by following links from a top-level hypertext document.
One implication is that we should provide links describing all
relationships and possible application state changes.
Another is that a GET on the base URL should return a document with
links to our various collections e.g.
$> GET /
<api>
<link rel="pools" href="/conductor/pools"/>
<link rel="deployments" href="/conductor/deployments"/>
...
</api>
= Authentication =
We currently support HTTP authentication just fine, but we should figure
out a way to avoid allocating cookies for such clients.
i.e. this seems strange:
GET /conductor/pools HTTP/1.1
Authorization: Basic YWRtaW46cGFzc3dvcmQ=
...
< HTTP/1.1 200 OK
...
< Set-Cookie: _rails23-app_session=8b0177c606d7de23437e5b13f5c8791f; path=/;
HttpOnly
= Create/Update Media Types =
For Create and Update, we should support XML and JSON. It should be
possible to take a representation of an existing resource, modify it and
use that to update or create a resource.
Currently, all our Create/Update operations accept x-www-form-urlencoded
as method bodies. We could consider supporting this as part of the API
but, if we do, we need to consider parameters like "commit=Save" and how
to support things like "provider[type]=ec2". It's perhaps easiest to
just say that this media type is an unsupported interface only intended
to be used by Conductors HTML and Javascript.
= Resource Creation =
Our current behaviour is:
POST /conductor/pools HTTP/1.1
Authorization: Basic YWRtaW46cGFzc3dvcmQ=
...
> Content-Type: application/x-www-form-urlencoded
} [data not shown]
< HTTP/1.1 302 Found
< Location:
http://localhost:3000/conductor/pools/3
...
i.e. we get redirected to the newly created pool.
This screws up curl which tries to follow the redirection and re-post.
We should return "HTTP 201 Created" and include a representation of the
created resource.