本文将结合SDM,浅谈下TLB。

1. TLB

1.1 Why

没有TLB的话,每次内存寻址时,都需要访问页表(页表位于内存中),开销较大。TLB缓存了VA(Virtual Address)到PA(Physical Address)的映射,当TLB hit时,就无需从内存中访问页表了。

1.2 What

The upper bits of a linear address (called the page number) determine the upper bits of the physical address (called the page frame); the lower bits of the linear address (called the page offset) determine the lower bits of the physical address. The boundary between the page number and the page offset is determined by the page size.

2. 全局TLB flush

每次进程切换时,需要更换CR3寄存器,flush掉全部的TLB Entries,开销较大。为此,硬件上引入了Global pagePCID机制,避免进程切换时flush掉所有的TLB Entries。

3. Global page

Linux内存管理中,内核空间是所有进程共享的,每个进程有自己独立的用户空间。进程切换时,是否可以不flush掉内核空间的TLB Entries呢?当然可以,Global page正是为此而生。

The Intel-64 and IA-32 architectures also allow for global pages when the PGE flag (bit 7) is 1 in CR4. If the G flag(bit 8) is 1 in a paging-structure entry that maps a page (either a PTE or a paging-structure entry in which the PS flag is 1), any TLB entry cached for a linear address using that paging-structure entry is considered to be global.

4. PCID

以前,是以(VA)这个一元组为key来唯一索引TLB Entry。PCID(Process-Context Identifier)的引入,硬件以(VA,进程信息)来唯一索引TLB Entry。这样,进程切换时,无需刷掉TLB Entries了,因为不同进程的(VA,进程信息)二元组是不同的。

4.1 Overlap between PCID and Global page

A logical processor may use a global TLB entry to translate a linear address, even if the TLB entry is associated with a PCID different from the current PCID.

5. TLB shootdown

6. VPID

虚拟化下,以(VA,进程信息)已经不能唯一索引TLB Entry了,因为不同virtual processors间,可能产生相同的(VA,进程信息)二元组,为此,在同一个物理CPU上,不同vCPU调度时,Hypervisor需要flush掉所有的TLB Entries。为此,VPID(Virtual-Processor IDentifier)应运而生。以(VA,进程信息,virtual processor信息)这个三元组来唯一索引TLB Entry。这样,在同一个物理CPU上,不同vCPU调度时,Hypervisor无需flush掉TLB Entries。

当然,VPID与PCID的使用不是绑定的,要看具体(Hypervisor和Guest OS)的实现了。例如,Hypervisor可以使用VPID,Guest OS不使用PCID,那么,此刻硬件是以(VA,virtual processor信息)来索引TLB Entry了。在同一个物理CPU上,不同vCPU调度时,Hypervisor无需flush掉TLB Entries,但是,在Guest OS中,每次进程切换时,Guest OS需要Flush掉TLB Entries来保证正确性。

7. Rethinking Protection Keys

之前已经介绍过了PKUPKS,这里,从TLB的视角,重新看下Protection Keys。

Memory Protection Keys (pkeys) are an extension to existing page-based memory permissions. Normal page permissions using page tables require expensive system calls and TLB invalidations when changing permissions. Memory Protection Keys provide a mechanism for changing protections without requiring modification of the page tables on every permission change.

Protection Keys是无需flush TLB的。留下的open是:

  1. 通过Protection Keys更改permissions,最终硬件是否会更改物理TLB Entry中的access rights位呢?
  2. 如果1成立的话,对于Multiple Processors,是否需要类似于propagating这个过程呢?这个过程是由软件or硬件来完成呢?

这些问题我当前并不知道答案,等待后续的更新。