Notes about RSS(Receive Side Scaling)

本文将mark下RSS(Receive Side Scaling)相关notes，内容主要转载自Introduction to Receive Side Scaling (RSS)。

1. Overview

A network interface card (NIC) can be configured with multiple receive queue (Rx). This is done to improve the throughput (i.e, the number of packets that can be processed per second) of the device and to achieve higher throughput, more than one CPU is used to poll(笔者注:这里指的是MSI-x中断收包) packets from the device independently. Each CPU polls(笔者注:这里指的是MSI-x中断收包) for its assigned Rx queue, but still we need a strategy to distribute the packets over all the queues, otherwise all the packets will go to one queue and only one CPU will be utilised.

2. Hash计算

2.1 RSS Input Fields

The RSS input fields plays a vital role in deciding the criteria for distributing the packets over multiple receive queue. There are lot of hash input fields available like source mac address, destination mac address, source IP, destination IP etc.

2.2 RSS Hash Key

The RSS hash is a 40/52 byte key used to randomize the distribution of packets over the receive queues. The size of the hash key depends on the hardware of the NIC.

2.3 Hash Algorithm

Once the hash function and hash key is configured, the hash algorithm is used to determine the receive queue index using the input fields and hash Key.

The hash functions which are generally used are:

Simple XOR Hash Function
Toeplitz Hash Function

2.4 hash计算总结

Incoming packets进入到parser模块，五元组src ip、dst ip、src port、dst port和protocol中的子集作为input set，结合一个任意的hash key以及hash function(默认hash function是Toeplitz function)计算得到hash value。

3. Working of RSS

When a NIC is started with multiple Rx queues then based on the configured RSS configurations like Input Fields, hash key and number of Rx queues, an indirection table is generated. An indirection table represents mapping between the hash value and the queue index.

When a packet is received by the NIC, all the values from the configured fields are extracted and those values are fed into the hash function. The generated hash value is used to do a lookup in the indirection table in order to get the destination queue index.

A number of least significant bits (LSBs) of the hash value are used to index an indirection table.

4. Advantage

There are two main advantages of using RSS:

Higher throughput as more number of CPUs can be used independently.
The computational cost of load balancing is offloaded to the NIC’s hardware instead of doing software based load-balancing.

5. Usage

RSS is the mechanism to process packets with multiple RX queues. When the network card with RSS receives packets, it will apply a filter to packets and distribute the packets to RX queues. The filter is usually a hash function and can be configured from ethtool -X. If you want to spread flows evenly among the first 3 queues:

1	ethtool -X eth0 equal 3

Or, if you find a magic hash key that is particularly useful:

1	ethtool -X eth0 hkey <magic hash key>

参考资料:

文章目录