Mellanox ConnectX(R) mlx5 core VPI Network Driver

Copyright (c) 2019, Mellanox Technologies LTD.

Enabling the driver and kconfig options

mlx5 core is modular and most of the major mlx5 core driver features can be selected (compiled in/out)
at build time via kernel Kconfig flags.
Basic features, ethernet net device rx/tx offloads and XDP, are available with the most basic flags
CONFIG_MLX5_CORE=y/m and CONFIG_MLX5_CORE_EN=y.
For the list of advanced features please see below.

CONFIG_MLX5_CORE=(y/m/n) (module mlx5_core.ko)

The driver can be enabled by choosing CONFIG_MLX5_CORE=y/m in kernel config.
This will provide mlx5 core driver for mlx5 ulps to interface with (mlx5e, mlx5_ib).

CONFIG_MLX5_CORE_EN=(y/n)

Choosing this option will allow basic ethernet netdevice support with all of the standard rx/tx offloads.
mlx5e is the mlx5 ulp driver which provides netdevice kernel interface, when chosen, mlx5e will be
built-in into mlx5_core.ko.

CONFIG_MLX5_EN_ARFS=(y/n)

Enables Hardware-accelerated receive flow steering (arfs) support, and ntuple filtering.

CONFIG_MLX5_EN_RXNFC=(y/n)

Enables ethtool receive network flow classification, which allows user defined
flow rules to direct traffic into arbitrary rx queue via ethtool set/get_rxnfc API.

CONFIG_MLX5_CORE_EN_DCB=(y/n):

CONFIG_MLX5_MPFS=(y/n)

Ethernet Multi-Physical Function Switch (MPFS) support in ConnectX NIC.
MPFs is required for when Multi-Host configuration is enabled to allow passing
user configured unicast MAC addresses to the requesting PF.

CONFIG_MLX5_ESWITCH=(y/n)

Ethernet SRIOV E-Switch support in ConnectX NIC. E-Switch provides internal SRIOV packet steering
and switching for the enabled VFs and PF in two available modes:

CONFIG_MLX5_CORE_IPOIB=(y/n)

IPoIB offloads & acceleration support.
Requires CONFIG_MLX5_CORE_EN to provide an accelerated interface for the rdma
IPoIB ulp netdevice.

CONFIG_MLX5_FPGA=(y/n)

Build support for the Innova family of network cards by Mellanox Technologies.
Innova network cards are comprised of a ConnectX chip and an FPGA chip on one board.
If you select this option, the mlx5_core driver will include the Innova FPGA core and allow
building sandbox-specific client drivers.

CONFIG_MLX5_EN_IPSEC=(y/n)

CONFIG_MLX5_EN_TLS=(y/n)

TLS cryptography-offload accelaration.

CONFIG_MLX5_INFINIBAND=(y/n/m) (module mlx5_ib.ko)

Provides low-level InfiniBand/RDMA and RoCE support.

External options ( Choose if the corresponding mlx5 feature is required )

  • CONFIG_PTP_1588_CLOCK: When chosen, mlx5 ptp support will be enabled
  • CONFIG_VXLAN: When chosen, mlx5 vxlan support will be enabled.
  • CONFIG_MLXFW: When chosen, mlx5 firmware flashing support will be enabled (via devlink and ethtool).

mlx5 tracepoints

mlx5 driver provides internal trace points for tracking and debugging using kernel tracepoints interfaces (refer to ftrace - Function Tracer).

For the list of support mlx5 events check /sys/kernel/debug/tracing/events/mlx5/

tc and eswitch offloads tracepoints:

  • mlx5e_configure_flower: trace flower filter actions and cookies offloaded to mlx5:

    $ echo mlx5:mlx5e_configure_flower >> /sys/kernel/debug/tracing/set_event
    $ cat /sys/kernel/debug/tracing/trace
    ...
    tc-6535  [019] ...1  2672.404466: mlx5e_configure_flower: cookie=0000000067874a55 actions= REDIRECT
    
  • mlx5e_delete_flower: trace flower filter actions and cookies deleted from mlx5:

    $ echo mlx5:mlx5e_delete_flower >> /sys/kernel/debug/tracing/set_event
    $ cat /sys/kernel/debug/tracing/trace
    ...
    tc-6569  [010] .N.1  2686.379075: mlx5e_delete_flower: cookie=0000000067874a55 actions= NULL
    
  • mlx5e_stats_flower: trace flower stats request:

    $ echo mlx5:mlx5e_stats_flower >> /sys/kernel/debug/tracing/set_event
    $ cat /sys/kernel/debug/tracing/trace
    ...
    tc-6546  [010] ...1  2679.704889: mlx5e_stats_flower: cookie=0000000060eb3d6a bytes=0 packets=0 lastused=4295560217
    
  • mlx5e_tc_update_neigh_used_value: trace tunnel rule neigh update value offloaded to mlx5:

    $ echo mlx5:mlx5e_tc_update_neigh_used_value >> /sys/kernel/debug/tracing/set_event
    $ cat /sys/kernel/debug/tracing/trace
    ...
    kworker/u48:4-8806  [009] ...1 55117.882428: mlx5e_tc_update_neigh_used_value: netdev: ens1f0 IPv4: 1.1.1.10 IPv6: ::ffff:1.1.1.10 neigh_used=1
    
  • mlx5e_rep_neigh_update: trace neigh update tasks scheduled due to neigh state change events:

    $ echo mlx5:mlx5e_rep_neigh_update >> /sys/kernel/debug/tracing/set_event
    $ cat /sys/kernel/debug/tracing/trace
    ...
    kworker/u48:7-2221  [009] ...1  1475.387435: mlx5e_rep_neigh_update: netdev: ens1f0 MAC: 24:8a:07:9a:17:9a IPv4: 1.1.1.10 IPv6: ::ffff:1.1.1.10 neigh_connected=1