xdp-forward is an XDP forwarding plane, which will accelerate packet forwarding using XDP. To use it, simply load it on the set of interfaces to accelerate forwarding between. The userspace component of xdp-forward will then configure and load XDP programs on those interfaces, and forward packets between them using XDP_REDIRECT, using the kernel routing table or netfilter flowtable to determine the destination for each packet.
Any packets that xdp-forward does not know how to forward will be passed up to the networking stack and handled by the kernel like normal. Depending on the mode xdp-forward is loaded in, this leads to different forwarding behaviours. See the sectinon on Operating modes below.
The syntax for running xdp-forward is:
xdp-forward COMMAND [options]
Where COMMAND can be one of:
load - Load the XDP forwarding plane
unload - Unload the XDP forwarding plane
help - show the list of available commandsEach command, and its options are explained below. Or use xdp-forward COMMAND
--help to see the options for each command.
The load command loads the XDP forwarding plane on a list of interfaces.
The syntax for the load command is:
xdp-forward load [options] <ifname...>
Where <ifname...> is the name of the set of interfaces to forward packets
between. An XDP program will be loaded on each interface, configured to forward
packets to all other interfaces in the set (using the kernel routing table to
determine the destination interface of each packet).
The supported options are:
Specifies which forwarding mode xdp-forward should operate in. Depending on
the mode selected, xdp-forward will perform forwarding in different ways,
which can lead to different behaviour, including which subset of kernel
configuration (such as firewall rules) is respected during forwarding. See the
section FORWARDING MODES below for a full description of each mode.
Specifies how xdp-forward performs routing table lookup in the linux kernel.
See the section FIB MODES below for a full description of each mode.
Specifies which mode to load the XDP program to be loaded in. The valid values are ‘native’, which is the default in-driver XDP mode, ‘skb’, which causes the so-called skb mode (also known as generic XDP) to be used, ‘hw’ which causes the program to be offloaded to the hardware, or ‘unspecified’ which leaves it up to the kernel to pick a mode (which it will do by picking native mode if the driver supports it, or generic mode otherwise). Note that using ‘unspecified’ can make it difficult to predict what mode a program will end up being loaded in. For this reason, the default is ‘native’. Note that hardware with support for the ‘hw’ mode is rare: Solarflare cards (using the ‘sfc’ driver) are the only devices with support for this in the mainline Linux kernel.
Enable debug logging. Specify twice for even more verbosity.
Display a summary of the available options
The unload command is used for unloading programs from an interface.
The syntax for the unload command is:
xdp-forward unload [options] <ifname...>
Where <ifname...> is the list of interfaces to unload the XDP forwarding plane
from. Note that while xdp-forward will examine the XDP programs loaded on each
interface and make sure to only unload its own program, it will not check that
the list of supplied interfaces is the same as the one supplied during load. As
such, it is possible to perform a partial unload by supplying a different list
of interfaces, which may lead to unexpected behaviour.
The supported options are:
Enable debug logging. Specify twice for even more verbosity.
Display a summary of the available options
The xdp-forward utility supports the following forwarding modes (selected by
the --fwd-mode parameter to xdp-forward load.
In the fib forwarding mode, xdp-forward will perform a lookup in
the kernel routing table (or FIB) for each packet, and forward packets between
the configured interfaces based on the result of the lookup. Any packet where
the lookup fails will be passed up to the stack. This includes packets that
require neighbour discovery for the next hop, meaning that packets will
periodically pass up the kernel stack for next hop discovery (initially, and
when the nexthop entry expires).
Note that no checks other than the FIB lookup is performed; in particular, this completely bypasses the netfilter subsystem, so firewall rules will not be checked before forwarding.
The flowtable operating mode offloads netfilter sw flowtable logic in
the XDP layer if the hardware flowtable is not available.
At the moment xdp-forward is able to offload just TCP or UDP netfilter
flowtable entries to XDP. The user is supposed to configure the flowtable
separately.
The xdp-forward utility supports the following fib modes (selected by
the --fib-mode parameter to xdp-forward load.
In the full operating mode, xdp-forward will perform a full lookup in
the kernel routing table (or FIB) for each packet, and forward packets between
the configured interfaces based on the result of the lookup. In particular,
it will apply any policy routing rules configured by the user.
The direct mode functions like full, except it passes the
BPF_FIB_LOOKUP_DIRECT flag to the FIB lookup routine. This means that any
policy routing rules configured will be skipped during the lookup, which can
improve performance (but won’t obey the policy of those rules, obviously).
In order to enable flowtable offloading for tcp and udp traffic between NICs n0 and n1, issue the following commands:
#nft -f /dev/stdin <<EOF
table inet filter {
flowtable ft {
hook ingress priority filter
devices = { n0, n1 }
}
chain forward {
type filter hook forward priority filter
meta l4proto { tcp, udp } flow add @ft
}
}
EOF
#xdp-forward load -f flowtable n0 n1libxdp(3) for details on the XDP loading semantics and kernel compatibility
requirements.
Please report any bugs on Github: https://github.com/xdp-project/xdp-tools/issues
xdp-forward is written by Toke Høiland-Jørgensen, based on the xdp_fwd kernel sample, which was originally written by David Ahern. This man page was written by Toke Høiland-Jørgensen.