Hey,
I started looking at this:
https://www.aeolusproject.org/redmine/issues/1200
"Use deltacloud api for provider names"
but decided to think through things from a higher level again.
I've a fairly jumbled set of notes below, but I'd summarize my
conclusions as:
1) We should move to using a single instance of deltacloud
2) We should drop current provider management UI in Conductor
3) The provider account UI should allow you to supply details of a
RHEV-M or VMWare instance that is not currently known to Conductor
or deltacloud. Or, indeed, choose from any of the known instances.
4) You shouldn't have to re-enter your EC2 credentials for each region
5) Image factory should use the provider/driver details from the
deltacloud API, once some more details are exposed
6) aeolus-image should be able to supply the RHEV-M or VMWare
instance URL to image_factory when doing a push; it gets this URL
from Conductor
7) Similarly, it looks like image_factory requires some other details
for each RHEV-M and VMWare instance. This would need to be in the
'Add Provider Account' dialog and passed by aeolus-image to
image_factory
Meh, that's quite a random set of work items. Anyone have any better
ideas?
Cheers,
Mark.
== Adding Provider Accounts =
In conductor, we need a UI for adding provider accounts.
This should be as straightforward as:
New Provider Account
Provider: [ mock v]
Username: mockuser
Password: ********
[ Add ]
for EC2:
New Provider Account
Provider: [ Amazon EC2 v]
Account number: 1234-5678-1337
Access key: 1DEADBEEF01234567890
Secret access key: ****************************************
Key: [ _________ ] [ Browse ]
Certificate: [ ________ ] [ Browse ]
[ Add ]
for RHEV-M:
New Provider Account
Provider: [ RHEV-M v]
( ) Use existing RHEV-M data center: [ default v]
(x) URL for other RHEV-M data center:
http://rhevm.mycorp.com/rhevm-api/
Username: markmc
Domain:
rhevm.mycorp.com
Password: ******
[ Add ]
Changes, improvements and rationale:
- When we have a single deltacloud instance for multiple providers, we
can reduce the importance of the "provider" concept in the UI. Only
provider accounts really matter any more
- deltacloud knows about all EC2 regions and automatically exposes a
provider for each one. See the /api/drivers collection and the
configuration in drivers.yaml describing each region/provider
- You only need to enter your credentials once and it is available for
use with all EC2 regions
- We can allow admins to add details of known RHEV-M or VMWare
instances so that they appear as an option. One simple way admins
can do that is by adding the RHEV-M instances as providers in
drivers.yaml, but we should probably have a conductor-level way of
doing it too
- Where deltacloud/conductor doesn't know anything about your RHEV-M
or VMWare instance, you can just enter its URL
- The credentials form is specific to each provider type
= Deltacloud Interface =
Deltacloud has a drivers.yaml file where it stores its knowledge about
different drivers and providers:
---
...
:mock:
:name: Mock
:rhevm:
:name: RHEVM
:entrypoints:
default:
default: "https://rhev-m.localhost:8443/rhevm-api-powershell"
...
:ec2:
:entrypoints:
...
ec2:
us-west-1:
ec2.us-west-1.amazonaws.com
ap-southeast-1:
ec2.ap-southeast-1.amazonaws.com
eu-west-1:
ec2.eu-west-1.amazonaws.com
us-east-1:
ec2.us-east-1.amazonaws.com
:name: EC2
Deltacloud can list all the providers it knows about through
the /api/drivers collection:
<drivers>
...
<driver href='http://localhost:3003/api/drivers/ec2' id='ec2'>
<name>EC2</name>
<provider id='us-west-1'></provider>
<provider id='ap-southeast-1'></provider>
<provider id='eu-west-1'></provider>
<provider id='us-east-1'></provider>
</driver>
...
<driver href='http://localhost:3003/api/drivers/rhevm'
id='rhevm'>
<name>RHEVM</name>
<provider id='default'></provider>
</driver>
</drivers>
With any deltacloud API call, you should be able to specify the driver
and provider using headers:
X-Deltacloud-Driver: rhevm
X-Deltacloud-Provider:
http://rhevm.mycorp.com/rhevm-api/
or matrix params:
GET /api;driver=ec2;provider=us-east-1/instances HTTP/1.1
Work needed:
- It would be nice to see the provider details in the /api/drivers
listing - e.g. the RHEV-M URLs, the EC2 endpoints etc.
- A simple change is needed to support having multiple RHEV-M URLs in
drivers.yaml and exposing them as different providers
- The driver= and provider= matrix params need to be URI encoded so
that we can pass a RHEV-M URL using the param
= Image Factory =
Getting to the original point of all this, image_factory has its own
notion of drivers/provider types/targets and providers.
Much of the information in deltacloud's drivers.yaml could form the
basis for image_factory's knowledge. For a start, we'd stop using
'rhevm' in deltacloud and 'rhev-m' in image_factory.
However, that doesn't account for providers added by Conductor when
deltacloud doesn't already know about the provider.
And finally, it also doesn't account for the other per-provider
knowledge that image_factory has, including:
- For EC2, the boto Location key for a given region:
https://github.com/boto/boto/blob/master/boto/s3/connection.py#L124
- For EC2, the i386 and x86_64 AMIs and AKIs IDs for Fedora 13 and 14
in each region
- For each RHEV-M provider, an NFS host and remote/local paths to use
for doing image uploads
- For VMWare providers, a 'VM host' which is used for image uploads
Some conclusions:
- image_factory doesn't depend on deltacloud right now. However, I
guess that in some far-flung future image_factory should be using
the deltacloud APIs rather than provider specific APIs. Therefore,
we shouldn't be afraid to add a dependency
- The easiest thing would be for image_factory to use deltacloud's
drivers.yaml directly but that's not an API, and may change format.
So, it makes sense that the info that image_factory requires should
from drivers.yaml should be exposed in the api/drivers collection.
- image_factory's other EC2 info should be in a config file, for easy
maintenance (i.e. the AMI/AKI IDs etc.)
- The per RHEV-M and VMWare instance info could be in an
image_factory config file, but we also need to handle the case
where the conductor user has added a provider account for a RHEV-M
instance not in the image_factory config
Perhaps the credentials XML should also contain the provider URL,
as well as the credentials?
- For the RHEV-M NFS details and VMWare 'VM host' details, we need to
consider whether they should be exposed in the conductor 'Add
Provider Account' UI or just in the image_factor config or, even,
automatically detected