There was some off-list discussion around the virtio-serial transport
for AMQP/qpid. I wanted to repaste the design here (provided by
ctrielof) so that we could continue discussion and hashing this out on
the upstream list.
Ok here is the basic setup.
We will create a 'virtio' transport plugin, by coping the ssl one and then
modifying it
and re factoring, and a final clean up. Once this is done I may also refactor away any
duplication, but we will focus first on getting the mechanics working correctly
The basic connect bring up working will be as follows.
There are two parts to an AMQP transport, the transport (plugin) for the broker, and the
transport plug-in for the AMQP client (in this case Matahari, post boot etc).
Flow:
The broker is run in the host, when the 'virtio' transport is loaded it will
'watch' for
guests to be started and open a domain socket to them using a well know addr. We do
this as we need the host (broker) to initiate the connections. Qemu can't initiate
eg. org.apache.qpid.amqp.0
This initial 'channel' is created as a control channel, or open listening
'port' in the guest
for any client to initiate communication to the broker.
Now when a Matahari process wants to connect to the AMQP broker, i.e. any AMQP client,
the client's virtio plug-in transport writes to the control channel
org.apache.qpid.amqp.virtio.0 and
requests a connection. The broker transport plug-in opens a new domain socket for this
connection
based on this request say org.apache.qpid.amqp.virtio.1 and returns this to the client
transport plugin
over org.apache.qpid.amqp.virtio.0
The client plug-in now closes the .0 channel, and connects to .1, and initiates the AMQP
communications as done in every other transport..
This means that we can now connect any and an arbitrary number of AMQP clients from the
guest to the broker over qemu.
There was also a follow up question regarding security (isolating guests
from each other):
sct commented:
How does security and guest isolation work if we've got a broker
connected to both the management plane and to the guest? We really need
a strong way to prevent the guest from getting free access to the
host-side management bus.
pmyers response:
guests can only have agents. agents can't query the bus, they
can only
publish events, provide objects with properties and methods. With the
ACL in place that states 'the virtio-serial transport only allows
entities connected to it in the guest to be Agents, and not Consoles'
you limit access to the bus and prevent a guest from being able to query
other guests or other entities outside of the host via the external
broker connections.
There was also discussion about naming of the devices on the host:
amit:
On the host we won't have such names. On the host, I prefer
/var/lib/matahari/guest-$GUEST-UUID/org.apache.qpid.amqp.0
The $GUEST-UUID can be obtained from libvirt's xml. This is to be used
to identify unique guests.
Alternatively, instead of UNIX sockets, tcp sockets can be used on the
host as well -- but then we need a port range to try connections to to
check if a guest is up.
(example: connect to tcp:localhost:5000-6000).
danpb commented:
Personally I always tend to recommend $NAME instead of $UUID for any
per-guest files that live on disk, because it is much more friendly
for an administrator to look at. All the files libvirt creates perguest
are based on $NAME.
I'd also not put them directly in /var/lib/matahari, but rather
do /var/lib/matahari/guest/$NAME/org.apache.qpid.amqp.0. This
means we can ensure that '/var/lib/matahari/guest/' directory can
get a suitable SELinux labelling to allow QEMU permission to create
the sockets, without conflicting with labelling we might have for
/var/lib/matahari due to other aspects of the system.