On 06/29/2011 05:11 AM, Chris Lalancette wrote:
On 06/28/11 - 03:12:02PM, Steven Dake wrote:
>>>> I had originally thought deltacloud might be involved here in some way
>>>> to consistently load instance data into the instance (and then provide a
>>>> consistent way of accessing it inside the vm that doesn't involve
>>>> writing 10 different access mechanisms).
>>>>
>>>> I don't know enough about deltacloud, but if the insertion
functionality
>>>> is currently a gap in any virtual machine manager (such as paravirt Xen)
>>>> or RHEVH, the entire Aeolus solution as well as cloud-ha becomes
>>>> non-functional on those VMMs.
>>>>
>>>> The question for deltacloud evaluation of this functionality then boils
>>>> down to:
>>>> 1) is this functionality available on every deltacloud cloud provider
>>>> implementation
>>>
>>> Not readily or consistently. For instance, rhev-m has "hook"
>>> functionality, but we haven't yet figured out what it means to use the
>>> hooks to provide this functionality for rhev-m.
>>>
>>> VMWare has some form of data injection available, as long as you install
>>> their guest agent, enabling certain api functionality that deltacloud
>>> could talk to.
>>>
>>> It's significantly different for each provider, that it takes research
>>> each time a new provider is taken on.
>>>
>>>> 2) is the functionality inside the vm consistent (does a user only call
>>>> a "read_my_id" function or read a file inside the vm) - if
library
>>>> access what is the new dependency inside the vm?
>>>
>>> No. In fact, for the case of EC2, this particularly breaks down.
>>> Because EC2 stores the "user data" in a place only accessible over
the
>>> network, the VM needs to ask a known ec2 location for the user data via
>>> an HTTP GET.
>>>
>>> Since dbus is probably already firmly set in stone by the time
>>> networking comes up, it's likely too late to rely on ec2's user data
to
>>> store to this information and have it impact dbus after retrieval.
>>
>> Just to be clear here, I don't know that this is the case. I haven't
>> personally looked into dbus, and how to configure the UUID on-the-fly. This
>> definitely bears investigation, as I feel that getting dbus to properly
>> expose this information is our best path forward.
>>
>
> looks like upstream is taking a crack at moving this into systemd rather
> then dbus........
>
>
http://lists.freedesktop.org/archives/dbus/2011-March/014187.html
>
> dbus does not have a set system id API.
>
> if we add a dbus set system id API and call it, say at the end of the
> startup sequence, some apps will be aware of the old id and others will
> be aware of the new id. I expect this would break things badly, but I
> am not a dbus expert so can't comment here.
>
> Perhaps we should let dbus do its own thing and just create a new file.
> Then we have two files
>
> /etc/machine_id (worthless in a cloud environment except for dbus operation)
> /etc/vm_machine_id (written by audrey or a dbus/systemd api) which can
> be set after the network is operational. Then the machine id logic can
> be put in one place (audrey) rather then a bunch of separate apps.
>
> I still have a bit of confusion how current VMs get their unique id
> inside a vm in the current model to communicate with Audrey's config server.
>
> more fuel on the fire:
>
http://www.freedesktop.org/wiki/Software/systemd/hostnamed
>
> Lets go back to basics, the requirements:
>
> R1: vm needs to have some mechanism of retrieving a globally unique UUID
> across all VMs in a cloud enviornment
> R2. whatever launches the vm needs to know how to map a UUID to an image
> R3. To obtain the vm UUID inside a vm, the network must be operational
> (is this true in atleast some cloud environments? If so, then it is a
> requirement).
Yes, this is all about right. Requirement 3 is the difficult one.
But maybe we are looking at this the wrong way. If we start from the premise
that dbus/systemd can generate it's own UUID at boot time (suitably
configured), then maybe we should think about using *that* as the UUID for
the machine. What this would then require is a mapping between the UUID
This would be a great option if anyone has any ideas how to generate
that mapping. The issue I get stuck on when thinking through this model
is how does the dbus id get out of the VM to make the mapping. The
problem is easy if only one vm starts at a time, but when multiple vms
start, I don't see a reasonable way to correlate the info.
This creates a different set of requirements:
1) dbus/systed must generate a fresh machine id on each boot
2) a translation service must exist to convert dbus ids to cloud ids.
1 seems challenging but solvable if we plan to run on older distros
(since these distros will have older versions of dbus/systemd). The id
currently isn't generated freshly on each boot.
One hacky workaround to 1 is the addition of an init script that deletes
the machine id file if it exists as the first thing that runs in the
system for older distros, and newer distros systemd could be configured
in some way to generate a fresh uuid via changes to their code base.
2 is more challenging - I don't have many ideas here. Audrey could
potentially determine the runtime vm UUID and send it to the config
server (which then could act as a translation service). This model
turns the audrey config service into a T component which from a design
standpoint is difficult to manage..
the cloud machinery has (i.e. the one that was generated prior to
launch) and
the UUID that dbus generated. It's a little ugly, but it prevents us from
having problems when we restart dbus/systemd on an already running system.
Again, I'm just throwing out ideas here. I still think we need to look closer
at dbus/systemd and see if we can possibly send some patches to those to
support changing the UUID in a cleaner manner. Then we can determine what the
best path forward is.
This leads to the same legacy problem - older distros wont have those
patch sets and audrey as well as pacemaker-cloud wont be operational on
those legacy distros.