Notes about VirtioVhostUser

https://mp.weixin.qq.com/s/hm5FVrcEsp19hlQny6euSA

1. What

The virtio-vhost-user device lets guests act as vhost device backends so that virtual network switches and storage appliance VMs can provide virtio devices to other guests.

virtio-vhost-user 设备可让客户机充当 vhost 设备后端，这样虚拟网络交换机和存储设备虚拟机就能为其他客户机提供 virtio 设备。

virtio-vhost-user was inspired by vhost-pci by Wei Wang and Zhiyong Yang.

virtio-vhost-user 的灵感来源于Wei Wang和Zhiyong Yang的 vhost-pci。

2. Use cases

2.1 Appliances for cloud environments

2.1 用于云环境的设备

In cloud environments everything is a guest. It is not possible for users to run vhost-user processes on the host. This precludes high-performance vhost-user appliances from running in cloud environments.

在云环境中，一切都是客户机。用户不可能在主机上运行 vhost-user进程。这使得高性能 vhost-user 设备无法在云环境中运行。

virtio-vhost-user allows vhost-user appliances to be shipped as virtual machine images. They can provide I/O services directly to other guests instead of going through an extra layer of device emulation like a host network switch:

virtio-vhost-user 允许 vhost-user 设备作为虚拟机镜像发布。它们可以直接向其他客户机提供 I/O 服务，而无需通过额外的设备仿真层（如主机网络交换机）：


    Traditional Appliance VMs       virtio-vhost-user Appliance VMs
+-------------+   +-------------+  +-------------+   +-------------+
|     VM1     |   |     VM2     |  |     VM1     |   |     VM2     |
|  Appliance  |   |   Consumer  |  |  Appliance  |   |   Consumer  |
|      ^      |   |      ^      |  |      <------+---+------>      |
+------|------+---+------|------+  +-------------+---+-------------+
|      +-----------------+      |  |                               |
|             Host              |  |             Host              |
+-------------------------------+  +-------------------------------+

2.2 Exitless VM-to-VM communication

Once the vhost-user session has been established all vring activity can be performed by poll mode drivers in shared memory. This eliminates vmexits in the data path so that the highest possible VM-to-VM communication performance can be achieved.

一旦 vhost-user 会话建立，所有 vring 活动都可由共享内存中的轮询模式驱动程序执行。这样就消除了数据路径中的 vmexits，从而实现尽可能高的VM-to-VM 通信性能。

Even when interrupts are necessary, virtio-vhost-user can use lightweight vmexits thanks to ioeventfd instead of exiting to host userspace. This ensures that VM-to-VM communication bypasses device emulation in QEMU.

即使需要中断(笔者注: virtio前端驱动kick需要发生VM Exit)，virtio-vhost-user 也可以通过 ioeventfd 使用轻量级 vmexits，而不是退出到主机用户空间。这可确保VM-to-VM 通信绕过 QEMU 中的设备仿真。

3. How it works

Virtio devices were originally emulated inside the QEMU host userspace process. Later on, vhost allowed a subset of a virtio device, called the vhost device backend, to be implement inside the host kernel. vhost-user then allowed vhost device backends to reside in host userspace processes instead.

Virtio 设备最初是在 QEMU 主机用户空间进程内仿真的。后来，vhost 允许一部分virtio 设备（称为 vhost 设备后端）在主机内核中实现。vhost-user 允许 vhost 设备后端驻留在主机用户空间进程中。

virtio-vhost-user takes this one step further by moving the vhost device backend into a guest. It works by tunneling the vhost-user protocol over a new virtio device type called virtio-vhost-user.

virtio-vhost-user 在此基础上更进一步，将 vhost 设备后端移至客户机中。它的工作原理是在名为 virtio-vhost-user 的新 virtio 设备类型上传输 vhost-user 协议。

The following diagram shows how two guests communicate:

下图显示了两个客户机的通信方式：


+-------------+                     +-------------+
|     VM1     |                     |     VM2     |
|             |                     |             |
|    vhost    |    shared memory    |             |
|   device    | +-----------------> |             |
|   backend   |                     |             |
|             |                     | virtio-net  |
+-------------+                     +-------------+
|             |                     |             |
|  virtio-    |  vhost-user socket  |             |
| vhost-user  | <-----------------> | vhost-user  |
|    QEMU     |                     |    QEMU     |
+-------------+                     +-------------+

VM2 sees a regular virtio-net device. VM2’s QEMU uses the existing vhost-user feature as if it were talking to a host userspace vhost-user backend.

VM2 看到的是普通的 virtio-net 设备。VM2 的 QEMU 使用现有的 vhost-user 功能，就像与主机用户空间 vhost-user 后端对话一样。

VM1’s QEMU tunnels the vhost-user protocol messages from VM1’s QEMU to the new virtio-vhost-user device so that guest software in VM1 can act as the vhost-user backend.

VM1 的 QEMU 将 vhost-user 协议信息从 VM1 的 QEMU 隧道(笔者注:可以类比于网络中的隧道技术，a method for transporting data across a network using protocols that are not supported by that network)传输到新的 virtio-vhost-user 设备，这样 VM1 中的客户机软件就可以充当 vhost-user 后端。

It is possible to reuse existing vhost-user backend software with virtio-vhost-user since they use the same vhost-user protocol messages. A driver is required for the virtio-vhost-user PCI device that carries the message instead of the usual vhost-user UNIX domain socket. The driver can be implemented in a guest userspace process using Linux vfio-pci but guest kernel driver implementation would also be also possible.

由于 virtio-vhost-user 使用相同的 vhost-user 协议信息，因此可以重新使用现有的 vhost-user 后端软件。virtio-vhost-user PCI 设备需要一个驱动程序来传输信息，而不是通常的 vhost-user UNIX 域套接字。该驱动程序可在客户机用户空间进程中使用 Linux vfio-pci 实现，也可在客户机内核驱动程序中实现。

The vhost device backend vrings are accessed through shared memory and do not require vhost-user message exchanges in the data path. No vmexits are taken when poll mode drivers are used. Even when interrupts are used, QEMU is not involved in the data path because ioeventfd lightweight vmexits are taken.

vhost 设备后端 vrings 通过共享内存访问，不需要在数据路径中进行 vhost 用户信息交换。使用轮询模式驱动程序时，不会出现 vmexits。即使使用中断(笔者注: virtio前端驱动kick需要发生VM Exit)，QEMU 也不会参与数据路径，因为会使用 ioeventfd 轻量级 vmexits。

All vhost device types work with virtio-vhost-user, including net, scsi, and blk.

所有 vhost 设备类型都能与 virtio-vhost-user 一起使用，包括 net、scsi 和 blk。

4. DPDK使用案例

下面截取了DPDK中VirtioVhostUser的使用案例:

slides中的Memory region I/O in device，笔者的理解就是VVU(VirtioVhostUser) device的MMIO寄存器。

参考资料:

文章目录