[vdsm] [RFC]about the implement of text-based console
Zhou Zheng Sheng
zhshzhou at linux.vnet.ibm.com
Tue Nov 27 09:22:20 UTC 2012
Hi all,
For now in there is no agreement on the remote guest console solution,
so I decide to do some investigation continue the discussion.
Our goal
VM serial console remote access in CLI mode. That means the client
runs without X environment.
There are several proposals.
1. Sandboxed sshd
VDSM runs a new host sshd instance in virtual machine/sandbox and
redirects the virtio console to it.
2. Third-party sshd
VDSM runs third-party sshd library/implementation and redirects
virtio console to it.
3. Spice
Extend spice to support console and implement a client to be run
without GUI environment
4. oVirt shell -> Engine -> libvirt
The user connects to Engine via oVirt CLI, then issues a
"serial-console" command, then Engine locates the host and connect to
the guest console. Currently there is a workaround, it invokes "virsh -c
qemu+tls://host/qemu console vmid" from Engine side.
5. VDSM console streaming API
VDSM exposes getConsoleReadStream() and getConsoleWriteStream() via
XMLRPC binding. Then implement the related client in vdsClient and Engine
Detailed discussion
1. Sandboxes
Solution 1 and 2 allow users connect to console using their favorite ssh
client. The login name is vmid, the password is set by setVmTicket()
call of VDSM. The connection will be lost during migration. This is
similar to VNC in oVirt.
I take a look at several sandbox technologies, including
libvirt-sandbox, lxc and selinux.
a) libvirt-sandbox boots a VM using host kernel and initramfs, then
passthru the host file system to the VM in read only mode. We can also
add extra binding to the guest file system. It's very easy to use. To
run shell in a VM, one can just issues
virt-sandbox -c qemu:///session /bin/sh
Then the VM will be ready in several seconds.
However it will trigger some selinux violations. Currently there is no
official support for selinux policy configuration from this project. In
the project page this is put in the todo list.
b) lxc utilize Linux container to run a process in sandbox. It needs to
be configured properly. I find in the package lxc-templates there is an
example configuration file for running sshd in lxc.
c) sandbox command in the package policycoreutils-python makes use of
selinux to run a process in sandbox, but there is no official or example
policy files for sshd.
In a word, for sandbox technologies, we have to configure the
policies/file system binding/network carefully and test the
compatibility with popular sshd implementations (openssh-server). When
those sshd upgrade, the policy must be upgraded by us at the same time.
Since the policies are not maintained by who implements sshd, this is a
burden for us.
Work to do
Write and maintain the policies.
Find ways for auth callback and redirecting data to openssh-server.
pros
Re-use existing pieces and technologies (host sshd, sandbox).
User friendly, they can use existing ssh clients.
cons
Connection is lost in migration, this is not a big problem because 1)
VNC connection share the same problem, 2) the user can reconnect manually.
It's not easy to maintain the sandbox policies/file system
binding/network for compatibility with sshd.
2. Third-party sshd implementations
Almost the same as solution 1 but with better flexibility. VDSM can
import a third-party sshd library and let that library deal with auth
and transport. VDSM just have to implement the data forwarding. Many
people consider this is insecure but I think the ticket solution for VNC
is even not as secure as this. Currently most of us only trust
openssh-server and think the quality of third-party sshd is low. I
searched for a while and found twisted.conch from the popular twisted
project. I'm not familiar with twisted.conch, but I still put it in this
mail to collect opinions from potential twisted.conch experts.
In a word, I prefer sandbox technologies to third-party sshd
implementations unless there is a implementation as good as openssh-server.
Work to do
Integrate twisted.conch into VDSM
pros
Very flexible. If library provide auth callback to VDSM, then VDSM
can just compares the login password to the VM ticket without knowing
SSH detials.
cons
Third party implementations are not as secure and carefully
maintained as sshd in the host (probably openssh-server).
3. Extend Spice to support console
Is it possible to implement a spice client can be run in pure text mode
without GUI environment? If we extend the protocol to support console
stream but the client must be run in GUI, it will be less useful.
pros
No new VMs and server process, easy for maintenance.
cons
Must wait for Spice developers to commit the support.
Need special client program in CLI, the user may prefer existing
client program like ssh. It not a big problem because this feature can
be put in to oVirt shell.
4. oVirt shell -> Engine -> libvirtd
This is the current workaround described in
http://wiki.ovirt.org/wiki/Features/Serial_Console_in_CLI#Currently_operational_workaround
The design is good but I do not like Engine talking to libvirtd
directly, thus comes the VDSM console streaming API below.
Work to do
Provide console streaming API from Engine to be invoked in oVirt shell.
Implement the "serial-console" command in oVirt shell.
pros
Support migration. Engine can reconnect to the guest automatically
after migration while keeping the connection from oVirt shell.
Fit well in the current oVirt architecture: no new server process
introduced, no new VM introduced, easy to maintain and manage.
cons
Engine talking to libvirtd directly breaks the encapsulation of VDSM.
Users only can get the console stream from Engine, they can not
directly connect to the host as VNC and the above two sshd solutions do.
5. VDSM console streaming API
Implement new APIs in VDSM to forward the raw data from console. It
exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC
binding. Then Engine can get the console data stream via API instead of
directly connecting to libvirtd. Other things will be the same as
solution 4.
Work to do
Implement getConsoleReadStream() and getConsoleWriteStream() in VDSM.
Provide console streaming API from Engine to be invoked in oVirt shell.
Implement the "serial-console" command in oVirt shell.
Optional: Implement a client program in vdsClient to consume the
stream API.
pros
Same as solution 4
cons
We can not allow ordinary user directly connect to VDSM and invoke
the stream API, because there is no ACL in VDSM, once a client cert is
setup for the ordinary user, he can call all the APIs in VDSM and get
total control. So the ordinary user can only get the stream from Engine,
and we leave Engine to do the ACL.
I like solution 4 best.
--
Thanks and best regards!
Zhou Zheng Sheng / 周征晟
E-mail: zhshzhou at linux.vnet.ibm.com
Telephone: 86-10-82454397
More information about the vdsm-devel
mailing list