On 06/28/2011 01:30 PM, Chris Lalancette wrote:
On 06/28/11 - 03:12:57PM, Greg Blomquist wrote:
> I talked to clalance for a moment on IRC to figure out what I wasn't
> seeing. The main piece was the bit between networking and dbus. Since
> dbus is likely to be up and configured by the time networking starts,
> there are limitations to how we can get to the "user data" in some
> cases. Specifically for ec2 (see comments below).
>> I had originally thought deltacloud might be involved here in some way
>> to consistently load instance data into the instance (and then provide a
>> consistent way of accessing it inside the vm that doesn't involve
>> writing 10 different access mechanisms).
>> I don't know enough about deltacloud, but if the insertion functionality
>> is currently a gap in any virtual machine manager (such as paravirt Xen)
>> or RHEVH, the entire Aeolus solution as well as cloud-ha becomes
>> non-functional on those VMMs.
>> The question for deltacloud evaluation of this functionality then boils
>> down to:
>> 1) is this functionality available on every deltacloud cloud provider
> Not readily or consistently. For instance, rhev-m has "hook"
> functionality, but we haven't yet figured out what it means to use the
> hooks to provide this functionality for rhev-m.
> VMWare has some form of data injection available, as long as you install
> their guest agent, enabling certain api functionality that deltacloud
> could talk to.
> It's significantly different for each provider, that it takes research
> each time a new provider is taken on.
>> 2) is the functionality inside the vm consistent (does a user only call
>> a "read_my_id" function or read a file inside the vm) - if library
>> access what is the new dependency inside the vm?
> No. In fact, for the case of EC2, this particularly breaks down.
> Because EC2 stores the "user data" in a place only accessible over the
> network, the VM needs to ask a known ec2 location for the user data via
> an HTTP GET.
> Since dbus is probably already firmly set in stone by the time
> networking comes up, it's likely too late to rely on ec2's user data to
> store to this information and have it impact dbus after retrieval.
Just to be clear here, I don't know that this is the case. I haven't
personally looked into dbus, and how to configure the UUID on-the-fly. This
definitely bears investigation, as I feel that getting dbus to properly
expose this information is our best path forward.
looks like upstream is taking a crack at moving this into systemd rather
dbus does not have a set system id API.
if we add a dbus set system id API and call it, say at the end of the
startup sequence, some apps will be aware of the old id and others will
be aware of the new id. I expect this would break things badly, but I
am not a dbus expert so can't comment here.
Perhaps we should let dbus do its own thing and just create a new file.
Then we have two files
/etc/machine_id (worthless in a cloud environment except for dbus operation)
/etc/vm_machine_id (written by audrey or a dbus/systemd api) which can
be set after the network is operational. Then the machine id logic can
be put in one place (audrey) rather then a bunch of separate apps.
I still have a bit of confusion how current VMs get their unique id
inside a vm in the current model to communicate with Audrey's config server.
more fuel on the fire:
Lets go back to basics, the requirements:
R1: vm needs to have some mechanism of retrieving a globally unique UUID
across all VMs in a cloud enviornment
R2. whatever launches the vm needs to know how to map a UUID to an image
R3. To obtain the vm UUID inside a vm, the network must be operational
(is this true in atleast some cloud environments? If so, then it is a
Our current limits:
systemd/dbus are nearly the first things that start and we can't teach
dbus about the vm UUID because of R3.