katello July 2011

katello@lists.fedorahosted.org

17 participants
17 discussions

Katello-Conductor-Factory-Warehouse interaction for 0.4.0 and beyond
by Hugh O. Brock 04 Aug '11

04 Aug '11

Hello all. We've had a lot of good discussion on the Aeolus list [1] about authorization and identity lately, and also about the intersection points of Aeolus and Katello. However I have been slow (sorry!) to suggest and update the requirements and user stories for this integration, and that has left folks asking a lot of questions about the features that are affected by that integration, in particular: * We will eventually need a UI for creating and managing templates, assemblies, and deployables -- where's that gonna go? * Precisely which components have to talk to each other securely in a system where Katello and Conductor work together? * Is there a need for an independent permission system in the Warehouse? What about an independent identity system? * One of the very cool features VCloud has is the ability to take a bunch of applications -- really, VM templates -- and put them in a "Catalog" that an admin can grant or revoke permission on for a set of users. The virtue of the catalog is that it gives admins an easy way to manage, say, "all the financial apps that are approved for production," or "the apps that are in testing this week." We need something like this for Aeolus, but how does it work exactly, and which part of the system owns it? I've been in a lot of meetings this week where we talked about these questions, so I'm going to propose some answers here based on that. There are a ton of unanswered questions as well, and I'm hopeful that Aeolus and Katello folks can help answer them here. (Katello folks, apologies for dumping a mostly-aeolus-centered design doc into your midst, but I thought you guys might want to know what we're thinking about too. Please feel free to comment on either or both lists.) 1. Template design and management ================================= (By "Template," I mean "Image templates, assembly definitions, and deployable definitions") We've had quite a bit of early user feedback asking, in no uncertain terms, for a nice GUI to let users define templates. We removed our nascent template definition UI from Conductor for the 0.3.0 release because it didn't feel right there, which I continue to think was the right idea, but there is clearly a need for such a GUI to exist somewhere. Fortunately, there is another project very close to Aeolus called Katello [2]. Katello's mission in life is allowing users to define bare-metal systems and manage their access to the software that should be provisioned on those systems. As such, it has as a core feature providing a UI for creating "System templates." I believe, and I think the Katello folks agree, that Katello users would benefit from being able to use it to define systems that will run in the cloud, as well as bare-metal systems. From there it's not a big leap to say that Katello users will want to define the entire range of templates that Aeolus can use and understand -- image templates, for stuff that gets built pre-boot; system templates (Katello) or assemblies (Audrey), for stuff that happens to a system post-boot; and application templates (Katello) or deployables (Audrey) for hooking systems together post-boot. Note that the last of these things -- application templates -- are also really useful for the bare-metal world. Given a set of system templates, and the ability to specify a set of systems that work together, it's not a big stretch to think that Katello users will want a way to provision and launch that set of systems -- an application -- either on a set of bare-metal machines, *or* in a cloud provided by Conductor. Of course, it will be some time before the Katello folks can design and build the whole UI for managing all three kinds of Aeolus templates and their interactions and dependencies. However, we don't have to wait for that to happen before we can start thinking about making Conductor and Katello work together. For release 0.4.0, I would think we could get by with the following features: * Katello provides an API that, given a user Katello knows about, will return all the Katello system templates that user has permission to see. (Note that templates in Katello are grouped by "environment," and that a user can have permission on more than one environment, so we'll have to deal with that hierarchy in the API somehow.) * The Aeolus team builds a UI piece in or next to Conductor will allow the user to: * map those templates to a catalog (more about this later) * map non-Katello templates (assemblies/deployables) to a catalog. For now this could be pretty much the same mechanism that our "Suggested Deployable" UI uses now -- store a URL of a deployable to use * Another UI piece, in or next to Conductor, will allow a user to: * map a catalog to a pool * check to see if the templates in that catalog have been built into images that have been pushed to the cloud provider accounts that the pool has access to * ask the Image Factory to build and push any templates that have *not* been built in the available accounts * display the status of the build(s) Somewhere in this process a translation step between the Katello system template and the Factory image template will need to happen. I don't particularly care where that step is. And it's important to note that eventually we will be passing Katello image templates to Factory, not Katello system templates -- hacking up the system template to get an image definition out of it is a temporary first step. I also think that in general, we should not be importing the actual template into a catalog, but rather just importing a reference to it so that we don't have duplicate copies of templates floating around. I'm willing to be argued out of this however. Some folks I talked to on the Katello side wondered if they should be able to tell something on the Aeolus side "Build this thing right now." I'm willing to say Conductor should provide a convenience API to do this, but it's important to note that Katello would need to be able to tell Conductor what pool (and hence what cloud providers) it wants the image built for, and provide a user who is authorized to push images to those providers. (One thing I notice as I write this is that we are going to need to pass user identity around fairly soon to accomplish this, which makes me think a central identity store is probably a firmer requirement for 0.4.0 than I initially thought.) So if we take the above feature list as "good enough" for 0.4.0, what does that tell us about who needs to talk to whom? 2. Secured communications ========================= I said above that we will have some UI in a catalog manager that hits a Katello API to retrieve all the system templates a user is allowed to see. For that to work, we're going to need to be able to give Katello a user; and for that to work, we're going to need a common identity store. Whether any of it is encrypted or not for this go-round is not terribly important I don't think, although of course it will have to be for production. I also said that the catalog manager will need to ask the Factory to build images for a set of cloud provider accounts. So there is going to have to be a channel to do that. However, I do not see anywhere here a user story that would require the catalog manager to act *on behalf of a user*; Conductor has already checked to see if the building user is allowed to build and push to the set of cloud provider accounts for the given pool. So I don't think the Factory needs to know anything about identity -- and, in fact, I don't think it really matters if non-Conductor users call the Factory and ask it to do stuff. Without the cloud provider account credentials, which are locked up in Conductor, there shouldn't be anything the user can do with the Factory that is harmful. Finally, I implied (without really saying so) that Conductor, or the catalog manager wherever that lives, needs to read information about the state of images from the warehouse, and push images to the warehouse. This means there will need to be a channel between the catalog manager and the Warehouse. However, I do not see anywhere here a user story that would require the catalog manager to talk to the Warehouse *on behalf of a user* - the ability to push, update, or launch images is all governed by Conductor's control of the cloud provider account credentials, so I don't think there is any reason for the Warehouse to have a notion of images belonging to users, or any understanding of user identity. Note that THIS IS A BIG CHANGE from what I and others have said in the past, but it does simplify what we have to do in the short term quite a bit. I believe the remaining secured communications -- between Conductor and Deltacloud, Conductor and Audrey, etc. -- are well enough understood that I don't have to discuss them here. So, to sum up: * Conductor (and its catalog manager) and Katello need to share user identity, and fairly soon * Conductor and Factory and Warehouse do not need to share user identity * In fact, Conductor and Katello are the *only* two components in this system that need to share identity. I realize I've just answered my third question above, about whether the Warehouse needs to know anything about authentication or authorization. I believe the answer is no. 3. Catalogs =========== Although my first thought about catalogs as a feature was "I guess we're doing this just because VMWare has them and we need to check the feature complete box," the more we talked about the feature the more I began to see it as a useful thing. Things administrators can do with catalogs: * For now, group image templates into a catalog; eventually, group application templates into a catalog * Connect catalogs to pools, making it possible for users to run the applications in the catalog * Check that the images that the applications in a catalog depend on, have been built for the cloud provider accounts connected to a pool, so that it is possible to launch the applications. * Disconnect catalogs from pools. Maybe, also remove images that are no longer referenced by an application that is connected to a pool. So from a user story perspective, you can imagine that an administrator, on creating or maintaining a pool, would be able to browse catalogs of applications to add to that pool. On adding the catalog to the pool, the admin would be able to check to make sure the images required by the apps in that catalog are built for all the provider accounts the pool has access to. The users of that pool would gain the ability to launch the listed applications. I would like to see the design activity for catalogs, and hopefully UX and wireframes, done soon. It would be really good to have a super-simple (just system templates) implementation by the 0.4.0 release. All right. That was long. I'll be waiting to hear how wrong I am from the assembled multitude. Please be thorough. Thanks, --Hugh [1] aeolus-devel(a)fedorahosted.org [2] http://katello.org -- == Hugh Brock, hbrock(a)redhat.com == == Engineering Manager, Cloud BU == == Aeolus Project: Manage virtual infrastructure across clouds. == == http://aeolusproject.org == "I know that you believe you understand what you think I said, but I’m not sure you realize that what you heard is not what I meant." --Robert McCloskey

7 16

Accessing katello
by Colin Coe 29 Jul '11

29 Jul '11

Hi all I've added this line --- -A INPUT -m state --state NEW -m tcp -p tcp --dport 3000 -j ACCEPT --- to my /etc/sysconfig/iptables file but I still can't bring up the site on my PC. netstat -anpe tells me that 0.0.0.0:3000 is LISTENing. Any ideas what else I need to do to get access (the website) from another computer? Thanks CC -- RHCE#805007969328369

2 1

Katello service is not starting
by Lukas Zapletal 28 Jul '11

28 Jul '11

Hello, due to bug in rack 1.1.0 the katello service is not working. The only way to start it until we fix this is: # cd /usr/lib/katello # su - katello -s /bin/bash -c "rails s -e production -p 3000" Please note there is no "-d" (daemon) option. It currently does not work. We are working hard to resolve permission errors. The next build will have all the assets and CSS files precompiled. -- Later, Lukas "lzap" Zapletal

4 6

Starting katello as root
by Lukas Zapletal 27 Jul '11

27 Jul '11

Hello, although you could run katello instance using these commands as root: # cd /usr/lib/katello # script/rails s -p 3000 -e production I do not recommend that. In this case all the temp files, pid file, database files (lock files) and even more in the development mode are created with the root owner. When you try to restart it with our init script (under "katello" user) you usually hit problems. Katello wont start without much error info. That's because it cannot sometimes even spawn a new thread, or it simply cannot create temp files. If you do that for whatever reason, keep running it as root forever. -- Later, Lukas "lzap" Zapletal

1 0

You must have at least one org
by Lukas Zapletal 26 Jul '11

26 Jul '11

Colin, starting a new thread. > I now see the login screen (using http not https). using the > default username/password emailed to the list, I see "You must have at > least one organiszation in your database to access that page. Might > need to run 'rake db:seed'" > So I ran 'rake db:seed' and it told me I needed to do a 'rake > db:migrate'. I did what it told me to do but I'm still being told I > need at least one org. # service katello initdb and restart -- Later, Lukas "lzap" Zapletal

4 5

Katello won't start
by Colin Coe 26 Jul '11

26 Jul '11

Hi all I had to rebuild my katello test VM and now, after following the install procedure, katello won't start. --- [root@fedora15 katello]# service katello restart Restarting katello (via systemctl): Job failed. See system logs and 'systemctl status' for details. [FAILED] [root@fedora15 log]# systemctl status katello.service katello.service - SYSV: Init script for katello Loaded: loaded (/etc/rc.d/init.d/katello) Active: failed since Tue, 19 Jul 2011 08:16:50 +0800; 3min 26s ago Process: 19629 ExecStart=/etc/rc.d/init.d/katello start (code=exited, status=1/FAILURE) CGroup: name=systemd:/system/katello.service [root@fedora15 log]# ls -l /var/log/katello/ total 0 -rw-r--r--. 1 katello katello 0 Jul 18 11:20 production_delayed_jobs.log -rw-r--r--. 1 katello katello 0 Jul 18 11:20 production.log [root@fedora15 log]# --- The log files in /var/log/katello haven't been touched since katello was installed. I can see that pulp-server, tomcat6, and candlepin are all running. Any pointers on tracking down the problem behind katello not starting? This node runs Fedora 15 (x86_64) on top of RHEV 2.2 Thanks CC -- RHCE#805007969328369

8 29

katello init script
by Colin Coe 21 Jul '11

21 Jul '11

Hi all looking at the init script, I see --- 141 su $KATELLO_USER -s /bin/bash -c "bundle install --without 'test development' --local" --- When I manually run this (making sure KATELLO_USER is 'katello') I get "Could not locate Gemfile" When I change the above to --- 141 su - $KATELLO_USER -s /bin/bash -c "bundle install --without 'test development' --local" --- I get the list of gemfiles. Should this be changed? Thanks CC -- RHCE#805007969328369

2 2

RHEL6 rpm dependency error
by Matthew Davis 19 Jul '11

19 Jul '11

I did a recently fresh install of RHEL6. Everything was smooth until I tried to install the katello pkg. I wasn't able to capture the original message, because I did a yum --skip-broken, and error scrolled off the screen. If you need the original error, lmk. I can rebuild the box. But the dependency error is @ http://fpaste.org/L4dF/ This machine is pointed to a local satellite, but it doesn't look like it's a problem with rpm's that come from rh channels. -- Matthew Davis, RHCA Solutions Architect O: 919.754.4380 M: 704.465.0530

3 5

Re: Candlepin installation - qe test?!
by Lukas Zapletal 19 Jul '11

19 Jul '11

On 07/19/2011 02:12 PM, Garik Khachikyan wrote: > Hi Lukas, > > So if I understand the point above correctly: > One need to be sure that uninstall & installing the requiring gems again > will keep the system(s) functional? > > Is there a real life scenario that could happen (happened) ? > > But of course it could be one of the tests to be performed (In scopes of > initscript tests or separately) - I'll handle that. The point here is to fetch all the resources that are expected on the gem server (all the gems in their current "locked" versions). You might need to delete rubygem cache for it. What we want to test here is real HTTP request. E.g. to download all the gems literally. Could improve the situation for now until we find better solution for that. Maybe some gemrepo checker or something... -- Later, Lukas "lzap" Zapletal

1 0

Candlepin installation
by Tomas Lestach 19 Jul '11

19 Jul '11

Hey guys, I tried to install Candlepin according to https://fedorahosted.org/candlepin/wiki/Setup Setup failed with: # /usr/share/candlepin/cpsetup Dropping candlepin database Creating candlepin database Loading candlepin schema Writing configuration file Creating CA private key password Creating CA private key Creating CA public key Creating CA certificate Waiting for tomcat to restart... ########## ERROR ############ Error running command: wget -qO- http://localhost:8080/candlepin/admin/init Status code: 2048 Command output: Traceback (most recent call last): File "/usr/share/candlepin/cpsetup", line 215, in <module> main(sys.argv[1:]) File "/usr/share/candlepin/cpsetup", line 207, in main run_command("wget -qO- http://localhost:8080/candlepin/admin/init") File "/usr/share/candlepin/cpsetup", line 34, in run_command raise Exception("Error running command") Exception: Error running command tomcat seems to be running: # /etc/init.d/tomcat6 status tomcat6 (pid 3993) is running...[ OK ] but doesn't seem to be very happy: SEVERE: Failed to load keystore type PKCS12 with path conf/keystore due to PKCS12 java.security.KeyStoreException: PKCS12 at java.security.KeyStore.getInstance(libgcj.so.10) ... Caused by: java.security.NoSuchAlgorithmException: Algorithm [PKCS12] of type [KeyStore] from provider [gnu.javax.security.auth.callback.GnuCallbacks: name=GNU-CALLBACKS version=2.1] is not found The full catalina.log is attached. Looks like something, what shall be installed isn't. Any hints? I have a rhel6.1 x86_64 system. Regards, Tomas -- Tomas Lestach RHN Satellite Engineering, Red Hat

5 9

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

katello July 2011