建议先阅读Linux中的tick模式和isolated CPU vs housekeeping CPU。

Dedicated instances are currently disturbed by unnecessary jitter due to the emulated lapic timers fire on the same pCPUs which vCPUs resident. There is no hardware virtual timer on Intel for guest like ARM. Both programming timer in guest and the emulated timer fires incur vmexits. This patchset tries to avoid vmexit which is incurred by the emulated timer fires in dedicated instance scenario.

When nohz_full is enabled in dedicated instances scenario, the unpinned timer will be moved to the nearest busy housekeepers after commit 9642d18eee2cd (nohz: Affine unpinned timers to housekeepers) and commit 444969223c8 (“sched/nohz: Fix affine unpinned timers mess”). However, KVM always makes lapic timer pinned to the pCPU which vCPU residents, the reason is explained by commit 61abdbe0 (kvm: x86: make lapic hrtimer pinned). Actually, these emulated timers can be offload to the housekeeping cpus since APICv is really common in recent years. The guest timer interrupt is injected by posted-interrupt which is delivered by housekeeping cpu once the emulated timer fires.

The host admin should fine tuned, e.g. dedicated instances scenario w/ nohz_full cover the pCPUs which vCPUs resident, several pCPUs surplus for busy housekeeping, disable mwait/hlt/pause vmexits to keep in non-root mode, ~3% redis performance benefit can be observed on Skylake server.

源码实现建议参考kvm performance optimization technologies, part two“Exitless timer”一节。

参考资料:

  1. KVM: LAPIC: Implement Exitless Timer
  2. Exitless timer
  3. Boosting Dedicated Instance via KVM Tax Cut