[vdsm] [RFC]about the implement of text-based console

Zhou Zheng Sheng zhshzhou at linux.vnet.ibm.com
Tue Nov 27 09:22:20 UTC 2012


Hi all,

For now in there is no agreement on the remote guest console solution, 
so I decide to do some investigation continue the discussion.

Our goal
   VM serial console remote access in CLI mode. That means the client 
runs without X environment.

There are several proposals.

1. Sandboxed sshd
   VDSM runs a new host sshd instance in virtual machine/sandbox and 
redirects the virtio console to it.
2. Third-party sshd
   VDSM runs third-party sshd library/implementation and redirects 
virtio console to it.
3. Spice
   Extend spice to support console and implement a client to be run 
without GUI environment
4. oVirt shell -> Engine -> libvirt
   The user connects to Engine via oVirt CLI, then issues a 
"serial-console" command, then Engine locates the host and connect to 
the guest console. Currently there is a workaround, it invokes "virsh -c 
qemu+tls://host/qemu console vmid" from Engine side.
5. VDSM console streaming API
   VDSM exposes getConsoleReadStream() and getConsoleWriteStream() via 
XMLRPC binding. Then implement the related client in vdsClient and Engine


Detailed discussion

1. Sandboxes
Solution 1 and 2 allow users connect to console using their favorite ssh 
client. The login name is vmid, the password is set by setVmTicket() 
call of VDSM. The connection will be lost during migration. This is 
similar to VNC in oVirt.

I take a look at several sandbox technologies, including 
libvirt-sandbox, lxc and selinux.
a) libvirt-sandbox boots a VM using host kernel and initramfs, then 
passthru the host file system to the VM in read only mode. We can also 
add extra binding to the guest file system. It's very easy to use. To 
run shell in a VM, one can just issues

virt-sandbox -c qemu:///session  /bin/sh

Then the VM will be ready in several seconds.
However it will trigger some selinux violations. Currently there is no 
official support for selinux policy configuration from this project. In 
the project page this is put in the todo list.

b) lxc utilize Linux container to run a process in sandbox. It needs to 
be configured properly. I find in the package lxc-templates there is an 
example configuration file for running sshd in lxc.

c) sandbox command in the package policycoreutils-python makes use of 
selinux to run a process in sandbox, but there is no official or example 
policy files for sshd.

In a word, for sandbox technologies, we have to configure the 
policies/file system binding/network carefully and test the 
compatibility with popular sshd implementations (openssh-server). When 
those sshd upgrade, the policy must be upgraded by us at the same time. 
Since the policies are not maintained by who implements sshd, this is a 
burden for us.

Work to do
   Write and maintain the policies.
   Find ways for auth callback and redirecting data to openssh-server.

pros
   Re-use existing pieces and technologies (host sshd, sandbox).
   User friendly, they can use existing ssh clients.
cons
   Connection is lost in migration, this is not a big problem because 1) 
VNC connection share the same problem, 2) the user can reconnect manually.
   It's not easy to maintain the sandbox policies/file system 
binding/network for compatibility with sshd.


2. Third-party sshd implementations
Almost the same as solution 1 but with better flexibility. VDSM can 
import a third-party sshd library and let that library deal with auth 
and transport. VDSM just have to implement the data forwarding. Many 
people consider this is insecure but I think the ticket solution for VNC 
is even not as secure as this. Currently most of us only trust 
openssh-server and think the quality of third-party sshd is low. I 
searched for a while and found twisted.conch from the popular twisted 
project. I'm not familiar with twisted.conch, but I still put it in this 
mail to collect opinions from potential twisted.conch experts.

In a word, I prefer sandbox technologies to third-party sshd 
implementations unless there is a implementation as good as openssh-server.

Work to do
   Integrate twisted.conch into VDSM

pros
   Very flexible. If library provide auth callback to VDSM, then VDSM 
can just compares the login password to the VM ticket without knowing 
SSH detials.
cons
   Third party implementations are not as secure and carefully 
maintained as sshd in the host (probably openssh-server).


3. Extend Spice to support console
Is it possible to implement a spice client can be run in pure text mode 
without GUI environment? If we extend the protocol to support console 
stream but the client must be run in GUI, it will be less useful.

pros
   No new VMs and server process, easy for maintenance.
cons
   Must wait for Spice developers to commit the support.
   Need special client program in CLI, the user may prefer existing 
client program like ssh. It not a big problem because this feature can 
be put in to oVirt shell.


4. oVirt shell -> Engine -> libvirtd
This is the current workaround described in

http://wiki.ovirt.org/wiki/Features/Serial_Console_in_CLI#Currently_operational_workaround

The design is good but I do not like Engine talking to libvirtd 
directly, thus comes the VDSM console streaming API below.

Work to do
   Provide console streaming API from Engine to be invoked in oVirt shell.
   Implement the "serial-console" command in oVirt shell.

pros
   Support migration. Engine can reconnect to the guest automatically 
after migration while keeping the connection from oVirt shell.
   Fit well in the current oVirt architecture: no new server process 
introduced, no new VM introduced, easy to maintain and manage.
cons
   Engine talking to libvirtd directly breaks the encapsulation of VDSM.
   Users only can get the console stream from Engine, they can not 
directly connect to the host as VNC and the above two sshd solutions do.


5. VDSM console streaming API
Implement new APIs in VDSM to forward the raw data from console. It 
exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC 
binding. Then Engine can get the console data stream via API instead of 
directly connecting to libvirtd. Other things will be the same as 
solution 4.

Work to do
   Implement getConsoleReadStream() and getConsoleWriteStream() in VDSM.
   Provide console streaming API from Engine to be invoked in oVirt shell.
   Implement the "serial-console" command in oVirt shell.
   Optional: Implement a client program in vdsClient to consume the 
stream API.

pros
   Same as solution 4
cons
   We can not allow ordinary user directly connect to VDSM and invoke 
the stream API, because there is no ACL in VDSM, once a client cert is 
setup for the ordinary user, he can call all the APIs in VDSM and get 
total control. So the ordinary user can only get the stream from Engine, 
and we leave Engine to do the ACL.

I like solution 4 best.

-- 
Thanks and best regards!

Zhou Zheng Sheng / 周征晟
E-mail: zhshzhou at linux.vnet.ibm.com
Telephone: 86-10-82454397



More information about the vdsm-devel mailing list