Hi all,
As we plan the next iteration, I've been asked to organize our work around the ability to launch an image without having to manually craft an XML file and host it on a webserver. Instead, Conductor users should be able to use a tool in Conductor to browse images stored in Image Warehouse, click one of them, select a hardware profile and maybe a pool, and launch it.
There's a semi-related feature to be able to import images. I've proposed treating it as a separate feature so as to not confuse the discussion, but it's also in scope for this next iteration so I'll mention it here.
My plan, if I haven't misunderstood the request, is to start a discussion here, and then create a wiki page once the discussion stabilizes a bit.
The work required is something like this:
* Figure out how this should interface with the UI. I think the actual "launch an instance" form can be very simple, but it's not abundantly clear to me where we should place the listing of images in Warehouse. We should solve this before we dive in on implementation.
* Interface with Image Warehouse. We're already doing this to some extent, but two major areas came up on the call today. The first is that a lot of what we're doing is already supported in the aeolus-image command-line tool. To avoid code duplication and reinventing the wheel, we'd like to break out a common library we can both use. (This also applies to image importation.) Obviously, we'll want to work with the Infrastructure team around aeolus-image. Second, we endeavor to switch from our custom-written WarehouseModel code using rest-client and nokogiri to ActiveResource.
* Actually implement it once we know where it goes, how it should look, and have a better layer to connect to Warehouse. We'll collect the user data, probably within the context of a pool, create a deployable behind the scenes with this image in it, and then launch it like it's a normal deployable.
This task should convey the following advantages:
* The workflow will be simplified for users who do not need multi-image deployable. They will not need to create an XML file, and will not be required to have a webserver running to host the deployable XML. I know from experience that the whole notion of pasting UUIDs into XML files and pushing them to webservers was not entirely popular among potential users.
* It will provide an interface for browsing Image Warehouse. Right now (as far as I'm aware), you're resigned to either getting XML back from Warehouse, or using the command-line tools if you want to know what images are stored in Conductor. Both are perfectly valid, but we'll implicitly provide a GUI.
* Ideally, we will incidentally improve the responsiveness of our interface to Warehouse as we switch to ActiveResource, though this remains to be seen.
* Although it's not an intentional goal, this may make it easier down the road if we ever decide that we want to provide a GUI for creating deployables.
Let the discussion commence!
-- Matt
Hi Matt,
On Tue, 2011-07-26 at 15:09 -0400, Matt Wagner wrote:
Hi all,
As we plan the next iteration, I've been asked to organize our work around the ability to launch an image without having to manually craft an XML file and host it on a webserver. Instead, Conductor users should be able to use a tool in Conductor to browse images stored in Image Warehouse, click one of them, select a hardware profile and maybe a pool, and launch it.
w00t! :)
What would the script for a screencast demo look like? e.g.
- Import an AMI by copy and pasting an AMI ID into an "Import Image" form
- You're forwarded to the images list, with your imported image highlighted
- Click on the "Launch" button beside the image
- You're forwarded to a page describing the instance. Wait for it to boot up
- Log into the image, fool around
- Shut it down again from instance's page
Thinking about this script is a good first start to getting the UX design right. With this script, the user is focused on "Launching an Image" so they're launching the instance from the images list. This shows that maybe a "New Instance" form perhaps isn't the right way to add the feature.
There's a semi-related feature to be able to import images. I've proposed treating it as a separate feature so as to not confuse the discussion, but it's also in scope for this next iteration so I'll mention it here.
I'm not so sure because:
1) This standalone instances feature is what's motivating the adding of the import image feature. From a user perspective, I'm not sure how interesting import image is on its own
2) If we go with the design above where users launch instances from the images list, a big part of what this feature needs will be the images list and the ability to import images
3) If import image was a standalone feature, I think we'd be tempted to overdo it - we might make it possible to import multiple provider images as an "Image" or "Image Family" and we might design we need to support the notion of image versions. None of that is really important right, and that's more obvious if you consider import image in the context of standalone instances
My plan, if I haven't misunderstood the request, is to start a discussion here, and then create a wiki page once the discussion stabilizes a bit.
Cool.
The work required is something like this:
- Figure out how this should interface with the UI. I think the actual "launch an instance" form can be very simple, but it's not abundantly clear to me where we should place the listing of images in Warehouse. We should solve this before we dive in on implementation.
Absolutely. You should take a first cut at thinking about the use cases and sketching the interface out yourself.
Interface with Image Warehouse. We're already doing this to some extent, but two major areas came up on the call today. The first is that a lot of what we're doing is already supported in the aeolus-image command-line tool. To avoid code duplication and reinventing the wheel, we'd like to break out a common library we can both use. (This also applies to image importation.) Obviously, we'll want to work with the Infrastructure team around aeolus-image. Second, we endeavor to switch from our custom-written WarehouseModel code using rest-client and nokogiri to ActiveResource.
Actually implement it once we know where it goes, how it should look, and have a better layer to connect to Warehouse. We'll collect the user data, probably within the context of a pool, create a deployable behind the scenes with this image in it, and then launch it like it's a normal deployable.
Hmm, I don't think we should treat standalone instances as deployments. The deployments interface is specifically suited to multiple instances. I think it would be more than a bit weird if "Launch Image" sent you to the deployments UI. An architecturally, I see deployments as being layered on top of the standalone instances.
This task should convey the following advantages:
- The workflow will be simplified for users who do not need multi-image deployable. They will not need to create an XML file, and will not be required to have a webserver running to host the deployable XML. I know from experience that the whole notion of pasting UUIDs into XML files and pushing them to webservers was not entirely popular among potential users.
Absolutely. Deployables are supposed to be reusable descriptions that someone has taken care to author and test. It's definitely not the best way to quickly launch one-off instances.
- It will provide an interface for browsing Image Warehouse. Right now (as far as I'm aware), you're resigned to either getting XML back from Warehouse, or using the command-line tools if you want to know what images are stored in Conductor. Both are perfectly valid, but we'll implicitly provide a GUI.
I'd be a little wary of implementing an "Image Warehouse Browser" e.g. showing images, builds, target images and provider images. This thing doesn't need to be generic, it just needs to offer the list of images that the user can launch.
Ideally, we will incidentally improve the responsiveness of our interface to Warehouse as we switch to ActiveResource, though this remains to be seen.
Although it's not an intentional goal, this may make it easier down the road if we ever decide that we want to provide a GUI for creating deployables.
Also:
* Support for standalone instances is needed for Conductor to implement deltacloud API
Nice one Matt, well done
Cheers, Mark.
Hi Mark,
I think we're envisioning slightly different things here, so I wanted to try to understand if I was unclear, or if you're describing a better way of doing things -- and if so, what the advantages are.
On Wed, Jul 27, 2011 at 10:30:35AM +0100, Mark McLoughlin wrote:
- If we go with the design above where users launch instances from the images list, a big part of what this feature needs will be the images list and the ability to import images
Yes, I agree that the features are fairly closely related. Mostly, I just wanted to break them apart into two smaller tasks -- give the ability to launch images from Warehouse, and give (err, bring back) the ability to import images using a tool in Conductor. They should both be done, though image importation isn't required for standalone launch.
- Actually implement it once we know where it goes, how it should look, and have a better layer to connect to Warehouse. We'll collect the user data, probably within the context of a pool, create a deployable behind the scenes with this image in it, and then launch it like it's a normal deployable.
Hmm, I don't think we should treat standalone instances as deployments.
The current UI is deployment-centric, though: we show pools which contain deployments, which contain instances.
If you launch a standalone image, I'm not sure why it should be treated any differently from the non-standalone images you've launched via deployables. It seems like we'd end up complicating the UI. Is there some advantage that I've overlooked to doing this?
The deployments interface is specifically suited to multiple instances.
I either misunderstand you or disagree with you here. Do you mean the deployment detail page? A deployment has many instances, so it can certainly support more than one instance. But everything works just as well with only one instance. I'm not sure why one instance should be a special case.
I'd be a little wary of implementing an "Image Warehouse Browser" e.g. showing images, builds, target images and provider images. This thing doesn't need to be generic, it just needs to offer the list of images that the user can launch.
Yes, agreed. I may have gone overboard with that description. I just want to make sure that, when we pull back the list of images that can be launched, we also pull back the other associated information that allows us to show more than a list of UUIDs.
Also:
- Support for standalone instances is needed for Conductor to implement deltacloud API
Oh, interesting. I hadn't even thought about this.
I'll work on getting this up onto the wiki.
-- Matt
On Wed, 2011-07-27 at 10:36 -0400, Matt Wagner wrote:
Hi Mark,
I think we're envisioning slightly different things here, so I wanted to try to understand if I was unclear, or if you're describing a better way of doing things -- and if so, what the advantages are.
On Wed, Jul 27, 2011 at 10:30:35AM +0100, Mark McLoughlin wrote:
- If we go with the design above where users launch instances from the images list, a big part of what this feature needs will be the images list and the ability to import images
Yes, I agree that the features are fairly closely related. Mostly, I just wanted to break them apart into two smaller tasks -- give the ability to launch images from Warehouse, and give (err, bring back) the ability to import images using a tool in Conductor. They should both be done, though image importation isn't required for standalone launch.
I would note that we already do this in aeolus-image _via factory), so it may be simpler/better to just require the import command from that lib and call it instead of writing up something new. If this means aolus-image needs any tweak, fine, but I think it should mostly 'just work' already. The same sentiment applies to what you said about listing things from iwhd - that is something aeolus-image already does, though we absolutely should share a common base model (Scott and I had talked about active model for both). I would lean toward not making this yet another gem though, I kind of think it can just go in aeolus-image if conductor is using it already anyway.
As an aside, the aeolus-image library was designed to either be used from cli (as we are at the moment) _or_ as an api for apps like conductor. In the latter case, you would just set it up as a regular gem dep and load the command object you are interested in using. You are perfectly free to pass in your config via your own yml/config file as well, provided it map to the expected options properly.
-j
On Thu, Jul 28, 2011 at 04:26:03PM -0400, Jason Guiditta wrote:
Yes, I agree that the features are fairly closely related. Mostly, I just wanted to break them apart into two smaller tasks -- give the ability to launch images from Warehouse, and give (err, bring back) the ability to import images using a tool in Conductor. They should both be done, though image importation isn't required for standalone launch.
I would note that we already do this in aeolus-image _via factory), so it may be simpler/better to just require the import command from that lib and call it instead of writing up something new.
Yes, actually, I was perhaps remiss in not contacting you about this. The goal here is to get Conductor to use a common library with aeolus_image if at all possible. To the extent that it's already implemented on your end, it's a piece of cake for us, though we may find places where it makes sense to split functionality out.
work' already. The same sentiment applies to what you said about listing things from iwhd - that is something aeolus-image already does, though we absolutely should share a common base model (Scott and I had talked about active model for both). I would lean toward not making this yet another gem though, I kind of think it can just go in aeolus-image if conductor is using it already anyway.
I think I agree completely here. I don't feel strongly about where the code should live, as much as that it shouldn't be duplicated (or worse, implemented from scratch twice) in both places. If we can treat aeolus-image as a library and just use that in Conductor, that sounds fantastic.
Should we (meaning the Infrastructure and Conductor teams) work together on getting this over to Active Resource, if that's not what you're already using? (It sounds like there may also be authentication-related changes coming at some point, too.)
As an aside, the aeolus-image library was designed to either be used from cli (as we are at the moment) _or_ as an api for apps like conductor. In the latter case, you would just set it up as a regular gem dep and load the command object you are interested in using. You are perfectly free to pass in your config via your own yml/config file as well, provided it map to the expected options properly.
I obviously have a lot to learn about how aeolus-image works, but I think this sounds great.
-- Matt
On 07/27/11 - 10:30:35AM, Mark McLoughlin wrote:
Hmm, I don't think we should treat standalone instances as deployments. The deployments interface is specifically suited to multiple instances. I think it would be more than a bit weird if "Launch Image" sent you to the deployments UI. An architecturally, I see deployments as being layered on top of the standalone instances.
I think I would have to agree with Matt here that we should *not* special case single instances. There is certainly a case to be made for making the simple things simple, and wrapping a single instance with a deployment complicates things without much benefit. However, it keeps our model consistent, which I think is important both for the codebase (no special cases) and for documenting/explaining to users. That is, users are going to have to grasp the concept of a deployable in order to use our system in any non-trivial way, so the fact that we expose everything as a deployable, including single instances, keeps this consistent.
As a side benefit, wrapping a single instance as a deployable also allows us to continue to use Audrey/Config Server as you do for multiple instances. The benefits of audrey for a single instance aren't as big as they are for multiple, but again it keeps things consistent and allows you to do the same things with single instances that you can for multiple instances.
On Tue, Jul 26, 2011 at 03:09:36PM -0400, Matt Wagner wrote:
My plan, if I haven't misunderstood the request, is to start a discussion here, and then create a wiki page once the discussion stabilizes a bit.
Not to imply that I think the discussion is over, but I went ahead and started a wiki page for this feature: https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Feature_-_Launch_...
I still need to brush it up, and break it out into tasks. But I wanted to get it onto the wiki even if more discussion needs to take place.
Incidentally, I took it upon myself to create a section on the main wiki page for these wiki pages: https://www.aeolusproject.org/redmine/projects/aeolus/wiki/
-- Matt
On Tue, 2011-07-26 at 15:09 -0400, Matt Wagner wrote:
Hi all,
As we plan the next iteration, I've been asked to organize our work around the ability to launch an image without having to manually craft an XML file and host it on a webserver. Instead, Conductor users should be able to use a tool in Conductor to browse images stored in Image Warehouse, click one of them, select a hardware profile and maybe a pool, and launch it.
There's a semi-related feature to be able to import images. I've proposed treating it as a separate feature so as to not confuse the discussion, but it's also in scope for this next iteration so I'll mention it here.
My plan, if I haven't misunderstood the request, is to start a discussion here, and then create a wiki page once the discussion stabilizes a bit.
The work required is something like this:
Figure out how this should interface with the UI. I think the actual "launch an instance" form can be very simple, but it's not abundantly clear to me where we should place the listing of images in Warehouse. We should solve this before we dive in on implementation.
Interface with Image Warehouse. We're already doing this to some extent, but two major areas came up on the call today. The first is that a lot of what we're doing is already supported in the aeolus-image command-line tool. To avoid code duplication and reinventing the wheel, we'd like to break out a common library we can both use. (This also applies to image importation.) Obviously, we'll want to work with the Infrastructure team around aeolus-image. Second, we endeavor to switch from our custom-written WarehouseModel code using rest-client and nokogiri to ActiveResource.
Curious on this decision - rest-client is able to return things in ActiveResource style, why is a switch needed? Or rather, what is the benefit? I ask, because we use rest-client in many aeolus components (including non-direct components, like deltacloud), and adding a different rest client adds a new dep, so I want to make sure we have a really good reason to do so after all the recent effort to decrease our number of dependencies.
-j
On Thu, Jul 28, 2011 at 04:28:58PM -0400, Jason Guiditta wrote:
- Interface with Image Warehouse. We're already doing this to some extent, but two major areas came up on the call today. The first is that a lot of what we're doing is already supported in the aeolus-image command-line tool. To avoid code duplication and reinventing the wheel, we'd like to break out a common library we can both use. (This also applies to image importation.) Obviously, we'll want to work with the Infrastructure team around aeolus-image. Second, we endeavor to switch from our custom-written WarehouseModel code using rest-client and nokogiri to ActiveResource.
Curious on this decision - rest-client is able to return things in ActiveResource style, why is a switch needed? Or rather, what is the benefit? I ask, because we use rest-client in many aeolus components (including non-direct components, like deltacloud), and adding a different rest client adds a new dep, so I want to make sure we have a really good reason to do so after all the recent effort to decrease our number of dependencies.
It's not so much that we want to swap one technology for another, as it is that we identified the current interface as being fairly roundabout and inefficient. We had hoped that switching to ActiveResource would clean this up. Perhaps this is erroneous.
That said, isn't ActiveResource a native component of Rails 3? Or is the goal to keep aeolus-image from requiring Rails?
-- Matt
It's not so much that we want to swap one technology for another, as it is that we identified the current interface as being fairly roundabout and inefficient. We had hoped that switching to ActiveResource would clean this up. Perhaps this is erroneous.
That said, isn't ActiveResource a native component of Rails 3? Or is the goal to keep aeolus-image from requiring Rails?
I haven't played around with it extensively but ActiveResource is part of the Rails project though it can be pulled in independently of the other components.
AFAIK ActiveResource gives us quite a bit more than simple invoking REST requests via rest-client. Deriving a class from ActiveResource::Base brings in a model-esque lifecycle to the data, eg we can perform CRUD operations on the object, setup relationships, add validations, and a make use of a slew of other features.
IMO we should be using ActiveResource more, we can derive alot of our current-model classes from ActiveResource::Base and drop them from the db inorder to prevent the data replication problem we often run into.
We can even derive Instance from ActiveResource if we are able to implement a REST frontend to condor. (possibly 0.4.0 feature?)
-Mo
aeolus-devel@lists.fedorahosted.org