Notes about KVM apf(async page fault)。转载自:KVM async page fault。

The qemu/kvm VM’s physical memory is the virtual memory of qemu process. When the virtual memory of qemu has been commit and is setup with physical memory the host can swap out this physical memory. When the guest vcpu access memory swapped out by host, its execution is suspended until memory is swapped back. Asynchronous page fault is a way to try and use guest vcpu more efficiently by allowing it to execute other tasks while page is brought back into memory. Following give a summary of these processes.

  1. page fault when the EPT page table is not setup

    1
    2
    3
    4
    5
    6
    7
    1. VMEXIT
    2. kvm_mmu_page_fault()
    3. gfn_to_pfn()
    4. get_user_pages_unlocked()
         no previously mapped page and no swap entry found
         empty page is allocated
    5. page is added into shadow/nested page table

  2. page fault when the physical memory is swapped out(without apf)

    1
    2
    3
    4
    5
    6
    7
    1. VMEXIT
    2. kvm_mmu_page_fault()
    3. gfn_to_pfn()
    4. get_user_pages_unlocked()
        swap entry is found
        page swap-in process is initiated
        vcpu thread goes to sleep until page is swapped in

  3. page fault when the phycial memory is swapped out(with apf)

    1
    2
    3
    4
    5
    6
    7
    8
    1. VMEXIT
    2. kvm_mmu_page_fault()
    3. gfn_to_pfn()
    4. get_user_pages_nowait()
    5. gup(get_user_pages) is done by dedicated thread, inject 'page not present' exception to guest
    6. guest puts process A(which caused this page fault) to sleep and schedule another process
    7. page is swapped in, inject 'page ready' exception to guest
    8. guest can schedule process A back to run on vcpu

Following shows the process of kvm async page fault process.

From description we know that kvm apf need the guest do something, such as recognize the apf ‘page not present’ and ‘page ready’ exception, and also the pv guest should hook the exception to process these two new exception. apf contains following steps.

1
2
3
4
5
1. the guest should be initialized to process the new exception
2. kvm page fault handler should recognize the swapped out case and initialize a work to swap in the page, inject a 'page not present' to guest
3. the guest receive this exception and schedule another process to run 
4. when the page caused page fault in step 2 has been swapped in, the kvm inject a 'page ready' exception to guest
5. the guest can do schedule to run process that was blocked by page fault in step 2

参考资料:

  1. KVM async page fault
  2. 2010-forum-Async-page-faults
  3. Add shared memory hypercall to PV Linux guest
  4. Qemu-kvm memory 虚拟化