€•¾TŒsphinx.addnodes”Œdocument”“”)”}”(Œ rawsource”Œ”Œchildren”]”(Œ translations”Œ LanguagesNode”“”)”}”(hhh]”(hŒ pending_xref”“”)”}”(hhh]”Œdocutils.nodes”ŒText”“”ŒChinese (Simplified)”…””}”Œparent”hsbaŒ attributes”}”(Œids”]”Œclasses”]”Œnames”]”Œdupnames”]”Œbackrefs”]”Œ refdomain”Œstd”Œreftype”Œdoc”Œ reftarget”Œ+/translations/zh_CN/networking/net_failover”Œmodname”NŒ classname”NŒ refexplicit”ˆuŒtagname”hhh ubh)”}”(hhh]”hŒChinese (Traditional)”…””}”hh2sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/zh_TW/networking/net_failover”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒItalian”…””}”hhFsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/it_IT/networking/net_failover”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒJapanese”…””}”hhZsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/ja_JP/networking/net_failover”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒKorean”…””}”hhnsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/ko_KR/networking/net_failover”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒPortuguese (Brazilian)”…””}”hh‚sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/pt_BR/networking/net_failover”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒSpanish”…””}”hh–sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/sp_SP/networking/net_failover”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubeh}”(h]”h ]”h"]”h$]”h&]”Œcurrent_language”ŒEnglish”uh1h hhŒ _document”hŒsource”NŒline”NubhŒcomment”“”)”}”(hŒ SPDX-License-Identifier: GPL-2.0”h]”hŒ SPDX-License-Identifier: GPL-2.0”…””}”hh·sbah}”(h]”h ]”h"]”h$]”h&]”Œ xml:space”Œpreserve”uh1hµhhh²hh³ŒE/var/lib/git/docbuild/linux/Documentation/networking/net_failover.rst”h´KubhŒsection”“”)”}”(hhh]”(hŒtitle”“”)”}”(hŒ NET_FAILOVER”h]”hŒ NET_FAILOVER”…””}”(hhÏh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÍhhÊh²hh³hÇh´KubhÉ)”}”(hhh]”(hÎ)”}”(hŒOverview”h]”hŒOverview”…””}”(hhàh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÍhhÝh²hh³hÇh´KubhŒ paragraph”“”)”}”(hŒåThe net_failover driver provides an automated failover mechanism via APIs to create and destroy a failover master netdev and manages a primary and standby slave netdevs that get registered via the generic failover infrastructure.”h]”hŒåThe net_failover driver provides an automated failover mechanism via APIs to create and destroy a failover master netdev and manages a primary and standby slave netdevs that get registered via the generic failover infrastructure.”…””}”(hhðh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´K hhÝh²hubhï)”}”(hXîThe failover netdev acts a master device and controls 2 slave devices. The original paravirtual interface is registered as 'standby' slave netdev and a passthru/vf device with the same MAC gets registered as 'primary' slave netdev. Both 'standby' and 'failover' netdevs are associated with the same 'pci' device. The user accesses the network interface via 'failover' netdev. The 'failover' netdev chooses 'primary' netdev as default for transmits when it is available with link up and running.”h]”hXThe failover netdev acts a master device and controls 2 slave devices. The original paravirtual interface is registered as ‘standby’ slave netdev and a passthru/vf device with the same MAC gets registered as ‘primary’ slave netdev. Both ‘standby’ and ‘failover’ netdevs are associated with the same ‘pci’ device. The user accesses the network interface via ‘failover’ netdev. The ‘failover’ netdev chooses ‘primary’ netdev as default for transmits when it is available with link up and running.”…””}”(hhþh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´KhhÝh²hubhï)”}”(hŒïThis can be used by paravirtual drivers to enable an alternate low latency datapath. It also enables hypervisor controlled live migration of a VM with direct attached VF by failing over to the paravirtual datapath when the VF is unplugged.”h]”hŒïThis can be used by paravirtual drivers to enable an alternate low latency datapath. It also enables hypervisor controlled live migration of a VM with direct attached VF by failing over to the paravirtual datapath when the VF is unplugged.”…””}”(hj h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´KhhÝh²hubeh}”(h]”Œoverview”ah ]”h"]”Œoverview”ah$]”h&]”uh1hÈhhÊh²hh³hÇh´KubhÉ)”}”(hhh]”(hÎ)”}”(hŒ-virtio-net accelerated datapath: STANDBY mode”h]”hŒ-virtio-net accelerated datapath: STANDBY mode”…””}”(hj%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÍhj"h²hh³hÇh´Kubhï)”}”(hŒšnet_failover enables hypervisor controlled accelerated datapath to virtio-net enabled VMs in a transparent manner with no/minimal guest userspace changes.”h]”hŒšnet_failover enables hypervisor controlled accelerated datapath to virtio-net enabled VMs in a transparent manner with no/minimal guest userspace changes.”…””}”(hj3h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´Khj"h²hubhï)”}”(hŒ®To support this, the hypervisor needs to enable VIRTIO_NET_F_STANDBY feature on the virtio-net interface and assign the same MAC address to both virtio-net and VF interfaces.”h]”hŒ®To support this, the hypervisor needs to enable VIRTIO_NET_F_STANDBY feature on the virtio-net interface and assign the same MAC address to both virtio-net and VF interfaces.”…””}”(hjAh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´K"hj"h²hubhï)”}”(hŒHHere is an example libvirt XML snippet that shows such configuration: ::”h]”hŒEHere is an example libvirt XML snippet that shows such configuration:”…””}”(hjOh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´K&hj"h²hubhŒ literal_block”“”)”}”(hX
”h]”hX
”…””}”hj_sbah}”(h]”h ]”h"]”h$]”h&]”hÅhÆuh1j]h³hÇh´K)hj"h²hubhï)”}”(hXIn this configuration, the first device definition is for the virtio-net interface and this acts as the 'persistent' device indicating that this interface will always be plugged in. This is specified by the 'teaming' tag with required attribute type having value 'persistent'. The link state for the virtio-net device is set to 'down' to ensure that the 'failover' netdev prefers the VF passthrough device for normal communication. The virtio-net device will be brought UP during live migration to allow uninterrupted communication.”h]”hX(In this configuration, the first device definition is for the virtio-net interface and this acts as the ‘persistent’ device indicating that this interface will always be plugged in. This is specified by the ‘teaming’ tag with required attribute type having value ‘persistent’. The link state for the virtio-net device is set to ‘down’ to ensure that the ‘failover’ netdev prefers the VF passthrough device for normal communication. The virtio-net device will be brought UP during live migration to allow uninterrupted communication.”…””}”(hjmh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´K;hj"h²hubhï)”}”(hX$The second device definition is for the VF passthrough interface. Here the 'teaming' tag is provided with type 'transient' indicating that this device may periodically be unplugged. A second attribute - 'persistent' is provided and points to the alias name declared for the virtio-net device.”h]”hX0The second device definition is for the VF passthrough interface. Here the ‘teaming’ tag is provided with type ‘transient’ indicating that this device may periodically be unplugged. A second attribute - ‘persistent’ is provided and points to the alias name declared for the virtio-net device.”…””}”(hj{h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´KChj"h²hubhï)”}”(hŒiBooting a VM with the above configuration will result in the following 3 interfaces created in the VM: ::”h]”hŒfBooting a VM with the above configuration will result in the following 3 interfaces created in the VM:”…””}”(hj‰h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´KHhj"h²hubj^)”}”(hX¹4: ens10: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff inet 192.168.12.53/24 brd 192.168.12.255 scope global dynamic ens10 valid_lft 42482sec preferred_lft 42482sec inet6 fe80::97d8:db2:8c10:b6d6/64 scope link valid_lft forever preferred_lft forever 5: ens10nsby: mtu 1500 qdisc fq_codel master ens10 state DOWN group default qlen 1000 link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff 7: ens11: mtu 1500 qdisc mq master ens10 state UP group default qlen 1000 link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff”h]”hX¹4: ens10: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff inet 192.168.12.53/24 brd 192.168.12.255 scope global dynamic ens10 valid_lft 42482sec preferred_lft 42482sec inet6 fe80::97d8:db2:8c10:b6d6/64 scope link valid_lft forever preferred_lft forever 5: ens10nsby: mtu 1500 qdisc fq_codel master ens10 state DOWN group default qlen 1000 link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff 7: ens11: mtu 1500 qdisc mq master ens10 state UP group default qlen 1000 link/ether 52:54:00:00:12:53 brd ff:ff:ff:ff:ff:ff”…””}”hj—sbah}”(h]”h ]”h"]”h$]”h&]”hÅhÆuh1j]h³hÇh´KLhj"h²hubhï)”}”(hŒ¡Here, ens10 is the 'failover' master interface, ens10nsby is the slave 'standby' virtio-net interface, and ens11 is the slave 'primary' VF passthrough interface.”h]”hŒ­Here, ens10 is the ‘failover’ master interface, ens10nsby is the slave ‘standby’ virtio-net interface, and ens11 is the slave ‘primary’ VF passthrough interface.”…””}”(hj¥h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´KWhj"h²hubhï)”}”(hXòOne point to note here is that some user space network configuration daemons like systemd-networkd, ifupdown, etc, do not understand the 'net_failover' device; and on the first boot, the VM might end up with both 'failover' device and VF acquiring IP addresses (either same or different) from the DHCP server. This will result in lack of connectivity to the VM. So some tweaks might be needed to these network configuration daemons to make sure that an IP is received only on the 'failover' device.”h]”hXþOne point to note here is that some user space network configuration daemons like systemd-networkd, ifupdown, etc, do not understand the ‘net_failover’ device; and on the first boot, the VM might end up with both ‘failover’ device and VF acquiring IP addresses (either same or different) from the DHCP server. This will result in lack of connectivity to the VM. So some tweaks might be needed to these network configuration daemons to make sure that an IP is received only on the ‘failover’ device.”…””}”(hj³h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´KZhj"h²hubhï)”}”(hŒbBelow is the patch snippet used with 'cloud-ifupdown-helper' script found on Debian cloud images::”h]”hŒeBelow is the patch snippet used with ‘cloud-ifupdown-helper’ script found on Debian cloud images:”…””}”(hjÁh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´Kbhj"h²hubj^)”}”(hXz@@ -27,6 +27,8 @@ do_setup() { local working="$cfgdir/.$INTERFACE" local final="$cfgdir/$INTERFACE" + if [ -d "/sys/class/net/${INTERFACE}/master" ]; then exit 0; fi + if ifup --no-act "$INTERFACE" > /dev/null 2>&1; then # interface is already known to ifupdown, no need to generate cfg log "Skipping configuration generation for $INTERFACE"”h]”hXz@@ -27,6 +27,8 @@ do_setup() { local working="$cfgdir/.$INTERFACE" local final="$cfgdir/$INTERFACE" + if [ -d "/sys/class/net/${INTERFACE}/master" ]; then exit 0; fi + if ifup --no-act "$INTERFACE" > /dev/null 2>&1; then # interface is already known to ifupdown, no need to generate cfg log "Skipping configuration generation for $INTERFACE"”…””}”hjÏsbah}”(h]”h ]”h"]”h$]”h&]”hÅhÆuh1j]h³hÇh´Kehj"h²hubeh}”(h]”Œ,virtio-net-accelerated-datapath-standby-mode”ah ]”h"]”Œ-virtio-net accelerated datapath: standby mode”ah$]”h&]”uh1hÈhhÊh²hh³hÇh´KubhÉ)”}”(hhh]”(hÎ)”}”(hŒBLive Migration of a VM with SR-IOV VF & virtio-net in STANDBY mode”h]”hŒBLive Migration of a VM with SR-IOV VF & virtio-net in STANDBY mode”…””}”(hjèh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÍhjåh²hh³hÇh´Kqubhï)”}”(hŒÏnet_failover also enables hypervisor controlled live migration to be supported with VMs that have direct attached SR-IOV VF devices by automatic failover to the paravirtual datapath when the VF is unplugged.”h]”hŒÏnet_failover also enables hypervisor controlled live migration to be supported with VMs that have direct attached SR-IOV VF devices by automatic failover to the paravirtual datapath when the VF is unplugged.”…””}”(hjöh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´Kshjåh²hubhï)”}”(hXHHere is a sample script that shows the steps to initiate live migration from the source hypervisor. Note: It is assumed that the VM is connected to a software bridge 'br0' which has a single VF attached to it along with the vnet device to the VM. This is not the VF that was passthrough'd to the VM (seen in the vf.xml file). ::”h]”hXKHere is a sample script that shows the steps to initiate live migration from the source hypervisor. Note: It is assumed that the VM is connected to a software bridge ‘br0’ which has a single VF attached to it along with the vnet device to the VM. This is not the VF that was passthrough’d to the VM (seen in the vf.xml file).”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´Kwhjåh²hubj^)”}”(hXR# cat vf.xml
# Source Hypervisor migrate.sh #!/bin/bash DOMAIN=vm-01 PF=ens6np0 VF=ens6v1 # VF attached to the bridge. VF_NUM=1 TAP_IF=vmtap01 # virtio-net interface in the VM. VF_XML=vf.xml MAC=52:54:00:00:12:53 ZERO_MAC=00:00:00:00:00:00 # Set the virtio-net interface up. virsh domif-setlink $DOMAIN $TAP_IF up # Remove the VF that was passthrough'd to the VM. virsh detach-device --live --config $DOMAIN $VF_XML ip link set $PF vf $VF_NUM mac $ZERO_MAC # Add FDB entry for traffic to continue going to the VM via # the VF -> br0 -> vnet interface path. bridge fdb add $MAC dev $VF bridge fdb add $MAC dev $TAP_IF master # Migrate the VM virsh migrate --live --persistent $DOMAIN qemu+ssh://$REMOTE_HOST/system # Clean up FDB entries after migration completes. bridge fdb del $MAC dev $VF bridge fdb del $MAC dev $TAP_IF master”h]”hXR# cat vf.xml
# Source Hypervisor migrate.sh #!/bin/bash DOMAIN=vm-01 PF=ens6np0 VF=ens6v1 # VF attached to the bridge. VF_NUM=1 TAP_IF=vmtap01 # virtio-net interface in the VM. VF_XML=vf.xml MAC=52:54:00:00:12:53 ZERO_MAC=00:00:00:00:00:00 # Set the virtio-net interface up. virsh domif-setlink $DOMAIN $TAP_IF up # Remove the VF that was passthrough'd to the VM. virsh detach-device --live --config $DOMAIN $VF_XML ip link set $PF vf $VF_NUM mac $ZERO_MAC # Add FDB entry for traffic to continue going to the VM via # the VF -> br0 -> vnet interface path. bridge fdb add $MAC dev $VF bridge fdb add $MAC dev $TAP_IF master # Migrate the VM virsh migrate --live --persistent $DOMAIN qemu+ssh://$REMOTE_HOST/system # Clean up FDB entries after migration completes. bridge fdb del $MAC dev $VF bridge fdb del $MAC dev $TAP_IF master”…””}”hjsbah}”(h]”h ]”h"]”h$]”h&]”hÅhÆuh1j]h³hÇh´K~hjåh²hubhï)”}”(hŒ¾On the destination hypervisor, a shared bridge 'br0' is created before migration starts, and a VF from the destination PF is added to the bridge. Similarly an appropriate FDB entry is added.”h]”hŒÂOn the destination hypervisor, a shared bridge ‘br0’ is created before migration starts, and a VF from the destination PF is added to the bridge. Similarly an appropriate FDB entry is added.”…””}”(hj h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´K¨hjåh²hubhï)”}”(hŒ¦The following script is executed on the destination hypervisor once migration completes, and it reattaches the VF to the VM and brings down the virtio-net interface::”h]”hŒ¥The following script is executed on the destination hypervisor once migration completes, and it reattaches the VF to the VM and brings down the virtio-net interface:”…””}”(hj.h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîh³hÇh´K¬hjåh²hubj^)”}”(hŒÔ# reattach-vf.sh #!/bin/bash bridge fdb del 52:54:00:00:12:53 dev ens36v0 bridge fdb del 52:54:00:00:12:53 dev vmtap01 master virsh attach-device --config --live vm01 vf.xml virsh domif-setlink vm01 vmtap01 down”h]”hŒÔ# reattach-vf.sh #!/bin/bash bridge fdb del 52:54:00:00:12:53 dev ens36v0 bridge fdb del 52:54:00:00:12:53 dev vmtap01 master virsh attach-device --config --live vm01 vf.xml virsh domif-setlink vm01 vmtap01 down”…””}”hj<sbah}”(h]”h ]”h"]”h$]”h&]”hÅhÆuh1j]h³hÇh´K°hjåh²hubeh}”(h]”Œ@live-migration-of-a-vm-with-sr-iov-vf-virtio-net-in-standby-mode”ah ]”h"]”ŒBlive migration of a vm with sr-iov vf & virtio-net in standby mode”ah$]”h&]”uh1hÈhhÊh²hh³hÇh´Kqubeh}”(h]”Œ net-failover”ah ]”h"]”Œ net_failover”ah$]”h&]”uh1hÈhhh²hh³hÇh´Kubeh}”(h]”h ]”h"]”h$]”h&]”Œsource”hÇuh1hŒcurrent_source”NŒ current_line”NŒsettings”Œdocutils.frontend”ŒValues”“”)”}”(hÍNŒ generator”NŒ datestamp”NŒ source_link”NŒ source_url”NŒ toc_backlinks”Œentry”Œfootnote_backlinks”KŒ sectnum_xform”KŒstrip_comments”NŒstrip_elements_with_classes”NŒ strip_classes”NŒ report_level”KŒ halt_level”KŒexit_status_level”KŒdebug”NŒwarning_stream”NŒ traceback”ˆŒinput_encoding”Œ utf-8-sig”Œinput_encoding_error_handler”Œstrict”Œoutput_encoding”Œutf-8”Œoutput_encoding_error_handler”j}Œerror_encoding”Œutf-8”Œerror_encoding_error_handler”Œbackslashreplace”Œ language_code”Œen”Œrecord_dependencies”NŒconfig”NŒ id_prefix”hŒauto_id_prefix”Œid”Œ dump_settings”NŒdump_internals”NŒdump_transforms”NŒdump_pseudo_xml”NŒexpose_internals”NŒstrict_visitor”NŒ_disable_config”NŒ_source”hÇŒ _destination”NŒ _config_files”]”Œ7/var/lib/git/docbuild/linux/Documentation/docutils.conf”aŒfile_insertion_enabled”ˆŒ raw_enabled”KŒline_length_limit”M'Œpep_references”NŒ pep_base_url”Œhttps://peps.python.org/”Œpep_file_url_template”Œpep-%04d”Œrfc_references”NŒ rfc_base_url”Œ&https://datatracker.ietf.org/doc/html/”Œ tab_width”KŒtrim_footnote_reference_space”‰Œsyntax_highlight”Œlong”Œ smart_quotes”ˆŒsmartquotes_locales”]”Œcharacter_level_inline_markup”‰Œdoctitle_xform”‰Œ docinfo_xform”KŒsectsubtitle_xform”‰Œ image_loading”Œlink”Œembed_stylesheet”‰Œcloak_email_addresses”ˆŒsection_self_link”‰Œenv”NubŒreporter”NŒindirect_targets”]”Œsubstitution_defs”}”Œsubstitution_names”}”Œrefnames”}”Œrefids”}”Œnameids”}”(jWjTjjjâjßjOjLuŒ nametypes”}”(jW‰j‰jâ‰jO‰uh}”(jThÊjhÝjßj"jLjåuŒ footnote_refs”}”Œ citation_refs”}”Œ autofootnotes”]”Œautofootnote_refs”]”Œsymbol_footnotes”]”Œsymbol_footnote_refs”]”Œ footnotes”]”Œ citations”]”Œautofootnote_start”KŒsymbol_footnote_start”KŒ id_counter”Œ collections”ŒCounter”“”}”…”R”Œparse_messages”]”hŒsystem_message”“”)”}”(hhh]”hï)”}”(hŒfPossible title underline, too short for the title. Treating it as ordinary text because it's so short.”h]”hŒhPossible title underline, too short for the title. Treating it as ordinary text because it’s so short.”…””}”(hjäh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hîhjáubah}”(h]”h ]”h"]”h$]”h&]”Œlevel”KŒtype”ŒINFO”Œline”K'Œsource”hÇuh1jßhj"h²hh³hÇh´K'ubaŒtransform_messages”]”Œ transformer”NŒ include_log”]”Œ decoration”Nh²hub.