€•Œsphinx.addnodes”Œdocument”“”)”}”(Œ rawsource”Œ”Œchildren”]”(Œ translations”Œ LanguagesNode”“”)”}”(hhh]”(hŒ pending_xref”“”)”}”(hhh]”Œdocutils.nodes”ŒText”“”ŒChinese (Simplified)”…””}”Œparent”hsbaŒ attributes”}”(Œids”]”Œclasses”]”Œnames”]”Œdupnames”]”Œbackrefs”]”Œ refdomain”Œstd”Œreftype”Œdoc”Œ reftarget”Œ+/translations/zh_CN/networking/snmp_counter”Œmodname”NŒ classname”NŒ refexplicit”ˆuŒtagname”hhh ubh)”}”(hhh]”hŒChinese (Traditional)”…””}”hh2sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/zh_TW/networking/snmp_counter”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒItalian”…””}”hhFsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/it_IT/networking/snmp_counter”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒJapanese”…””}”hhZsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/ja_JP/networking/snmp_counter”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒKorean”…””}”hhnsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/ko_KR/networking/snmp_counter”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒPortuguese (Brazilian)”…””}”hh‚sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/pt_BR/networking/snmp_counter”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒSpanish”…””}”hh–sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/sp_SP/networking/snmp_counter”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubeh}”(h]”h ]”h"]”h$]”h&]”Œcurrent_language”ŒEnglish”uh1h hhŒ _document”hŒsource”NŒline”NubhŒsection”“”)”}”(hhh]”(hŒtitle”“”)”}”(hŒ SNMP counter”h]”hŒ SNMP counter”…””}”(hh¼h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhh·h²hh³ŒE/var/lib/git/docbuild/linux/Documentation/networking/snmp_counter.rst”h´KubhŒ paragraph”“”)”}”(hŒ4This document explains the meaning of SNMP counters.”h]”hŒ4This document explains the meaning of SNMP counters.”…””}”(hhÍh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khh·h²hubh¶)”}”(hhh]”(h»)”}”(hŒGeneral IPv4 counters”h]”hŒGeneral IPv4 counters”…””}”(hhÞh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhhÛh²hh³hÊh´KubhÌ)”}”(hŒ•All layer 4 packets and ICMP packets will change these counters, but these counters won't be changed by layer 2 packets (such as STP) or ARP packets.”h]”hŒ—All layer 4 packets and ICMP packets will change these counters, but these counters won’t be changed by layer 2 packets (such as STP) or ARP packets.”…””}”(hhìh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K hhÛh²hubhŒ bullet_list”“”)”}”(hhh]”hŒ list_item”“”)”}”(hŒ IpInReceives ”h]”hÌ)”}”(hŒ IpInReceives”h]”hŒ IpInReceives”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K hjubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhhüh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œbullet”Œ*”uh1húh³hÊh´K hhÛh²hubhÌ)”}”(hŒ"Defined in `RFC1213 ipInReceives`_”h]”(hŒ Defined in ”…””}”(hj!h²hh³Nh´NubhŒ reference”“”)”}”(hŒ`RFC1213 ipInReceives`_”h]”hŒRFC1213 ipInReceives”…””}”(hj+h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipInReceives”Œrefuri”Œ+https://tools.ietf.org/html/rfc1213#page-26”uh1j)hj!Œresolved”Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KhhÛh²hubhŒtarget”“”)”}”(hŒE.. _RFC1213 ipInReceives: https://tools.ietf.org/html/rfc1213#page-26”h]”h}”(h]”Œrfc1213-ipinreceives”ah ]”h"]”Œrfc1213 ipinreceives”ah$]”h&]”j;j<uh1jDh´KhhÛh²hh³hÊŒ referenced”KubhÌ)”}”(hXYThe number of packets received by the IP layer. It gets increasing at the beginning of ip_rcv function, always be updated together with IpExtInOctets. It will be increased even if the packet is dropped later (e.g. due to the IP header is invalid or the checksum is wrong and so on). It indicates the number of aggregated segments after GRO/LRO.”h]”hXYThe number of packets received by the IP layer. It gets increasing at the beginning of ip_rcv function, always be updated together with IpExtInOctets. It will be increased even if the packet is dropped later (e.g. due to the IP header is invalid or the checksum is wrong and so on). It indicates the number of aggregated segments after GRO/LRO.”…””}”(hjSh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KhhÛh²hubhû)”}”(hhh]”j)”}”(hŒ IpInDelivers ”h]”hÌ)”}”(hŒ IpInDelivers”h]”hŒ IpInDelivers”…””}”(hjhh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khjdubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjah²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KhhÛh²hubhÌ)”}”(hŒ"Defined in `RFC1213 ipInDelivers`_”h]”(hŒ Defined in ”…””}”(hj‚h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipInDelivers`_”h]”hŒRFC1213 ipInDelivers”…””}”(hjŠh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipInDelivers”j;Œ+https://tools.ietf.org/html/rfc1213#page-28”uh1j)hj‚j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KhhÛh²hubjE)”}”(hŒE.. _RFC1213 ipInDelivers: https://tools.ietf.org/html/rfc1213#page-28”h]”h}”(h]”Œrfc1213-ipindelivers”ah ]”h"]”Œrfc1213 ipindelivers”ah$]”h&]”j;jšuh1jDh´KhhÛh²hh³hÊjRKubhÌ)”}”(hŒÿThe number of packets delivers to the upper layer protocols. E.g. TCP, UDP, ICMP and so on. If no one listens on a raw socket, only kernel supported protocols will be delivered, if someone listens on the raw socket, all valid IP packets will be delivered.”h]”hŒÿThe number of packets delivers to the upper layer protocols. E.g. TCP, UDP, ICMP and so on. If no one listens on a raw socket, only kernel supported protocols will be delivered, if someone listens on the raw socket, all valid IP packets will be delivered.”…””}”(hj­h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K hhÛh²hubhû)”}”(hhh]”j)”}”(hŒIpOutRequests ”h]”hÌ)”}”(hŒ IpOutRequests”h]”hŒ IpOutRequests”…””}”(hjÂh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K%hj¾ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj»h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´K%hhÛh²hubhÌ)”}”(hŒ#Defined in `RFC1213 ipOutRequests`_”h]”(hŒ Defined in ”…””}”(hjÜh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipOutRequests`_”h]”hŒRFC1213 ipOutRequests”…””}”(hjäh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipOutRequests”j;Œ+https://tools.ietf.org/html/rfc1213#page-28”uh1j)hjÜj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K'hhÛh²hubjE)”}”(hŒF.. _RFC1213 ipOutRequests: https://tools.ietf.org/html/rfc1213#page-28”h]”h}”(h]”Œrfc1213-ipoutrequests”ah ]”h"]”Œrfc1213 ipoutrequests”ah$]”h&]”j;jôuh1jDh´K)hhÛh²hh³hÊjRKubhÌ)”}”(hŒŽThe number of packets sent via IP layer, for both single cast and multicast packets, and would always be updated together with IpExtOutOctets.”h]”hŒŽThe number of packets sent via IP layer, for both single cast and multicast packets, and would always be updated together with IpExtOutOctets.”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K+hhÛh²hubhû)”}”(hhh]”j)”}”(hŒ!IpExtInOctets and IpExtOutOctets ”h]”hÌ)”}”(hŒ IpExtInOctets and IpExtOutOctets”h]”hŒ IpExtInOctets and IpExtOutOctets”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K/hjubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´K/hhÛh²hubhÌ)”}”(hX?They are Linux kernel extensions, no RFC definitions. Please note, RFC1213 indeed defines ifInOctets and ifOutOctets, but they are different things. The ifInOctets and ifOutOctets include the MAC layer header size but IpExtInOctets and IpExtOutOctets don't, they only include the IP layer header and the IP layer data.”h]”hXAThey are Linux kernel extensions, no RFC definitions. Please note, RFC1213 indeed defines ifInOctets and ifOutOctets, but they are different things. The ifInOctets and ifOutOctets include the MAC layer header size but IpExtInOctets and IpExtOutOctets don’t, they only include the IP layer header and the IP layer data.”…””}”(hj6h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K1hhÛh²hubhû)”}”(hhh]”j)”}”(hŒBIpExtInNoECTPkts, IpExtInECT1Pkts, IpExtInECT0Pkts, IpExtInCEPkts ”h]”hÌ)”}”(hŒAIpExtInNoECTPkts, IpExtInECT1Pkts, IpExtInECT0Pkts, IpExtInCEPkts”h]”hŒAIpExtInNoECTPkts, IpExtInECT1Pkts, IpExtInECT0Pkts, IpExtInCEPkts”…””}”(hjKh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K7hjGubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjDh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´K7hhÛh²hubhÌ)”}”(hŒ|They indicate the number of four kinds of ECN IP packets, please refer `Explicit Congestion Notification`_ for more details.”h]”(hŒGThey indicate the number of four kinds of ECN IP packets, please refer ”…””}”(hjeh²hh³Nh´Nubj*)”}”(hŒ#`Explicit Congestion Notification`_”h]”hŒ Explicit Congestion Notification”…””}”(hjmh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”Œ Explicit Congestion Notification”j;Œ*https://tools.ietf.org/html/rfc3168#page-6”uh1j)hjej=KubhŒ for more details.”…””}”(hjeh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K9hhÛh²hubjE)”}”(hŒP.. _Explicit Congestion Notification: https://tools.ietf.org/html/rfc3168#page-6”h]”h}”(h]”Œ explicit-congestion-notification”ah ]”h"]”Œ explicit congestion notification”ah$]”h&]”j;j}uh1jDh´KhhÛh²hubhû)”}”(hhh]”j)”}”(hŒIpInHdrErrors ”h]”hÌ)”}”(hŒ IpInHdrErrors”h]”hŒ IpInHdrErrors”…””}”(hj©h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KChj¥ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj¢h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KChhÛh²hubhÌ)”}”(hŒšDefined in `RFC1213 ipInHdrErrors`_. It indicates the packet is dropped due to the IP header error. It might happen in both IP input and IP forward paths.”h]”(hŒ Defined in ”…””}”(hjÃh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipInHdrErrors`_”h]”hŒRFC1213 ipInHdrErrors”…””}”(hjËh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipInHdrErrors”j;Œ+https://tools.ietf.org/html/rfc1213#page-27”uh1j)hjÃj=KubhŒw. It indicates the packet is dropped due to the IP header error. It might happen in both IP input and IP forward paths.”…””}”(hjÃh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KEhhÛh²hubjE)”}”(hŒF.. _RFC1213 ipInHdrErrors: https://tools.ietf.org/html/rfc1213#page-27”h]”h}”(h]”Œrfc1213-ipinhdrerrors”ah ]”h"]”Œrfc1213 ipinhdrerrors”ah$]”h&]”j;jÛuh1jDh´KIhhÛh²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒIpInAddrErrors ”h]”hÌ)”}”(hŒIpInAddrErrors”h]”hŒIpInAddrErrors”…””}”(hjùh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KKhjõubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjòh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KKhhÛh²hubhÌ)”}”(hŒÂDefined in `RFC1213 ipInAddrErrors`_. It will be increased in two scenarios: (1) The IP address is invalid. (2) The destination IP address is not a local address and IP forwarding is not enabled”h]”(hŒ Defined in ”…””}”(hjh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipInAddrErrors`_”h]”hŒRFC1213 ipInAddrErrors”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipInAddrErrors”j;Œ+https://tools.ietf.org/html/rfc1213#page-27”uh1j)hjj=KubhŒž. It will be increased in two scenarios: (1) The IP address is invalid. (2) The destination IP address is not a local address and IP forwarding is not enabled”…””}”(hjh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KMhhÛh²hubjE)”}”(hŒG.. _RFC1213 ipInAddrErrors: https://tools.ietf.org/html/rfc1213#page-27”h]”h}”(h]”Œrfc1213-ipinaddrerrors”ah ]”h"]”Œrfc1213 ipinaddrerrors”ah$]”h&]”j;j+uh1jDh´KQhhÛh²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒIpExtInNoRoutes ”h]”hÌ)”}”(hŒIpExtInNoRoutes”h]”hŒIpExtInNoRoutes”…””}”(hjIh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KShjEubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjBh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KShhÛh²hubhÌ)”}”(hXThis counter means the packet is dropped when the IP stack receives a packet and can't find a route for it from the route table. It might happen when IP forwarding is enabled and the destination IP address is not a local address and there is no route for the destination IP address.”h]”hXThis counter means the packet is dropped when the IP stack receives a packet and can’t find a route for it from the route table. It might happen when IP forwarding is enabled and the destination IP address is not a local address and there is no route for the destination IP address.”…””}”(hjch²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KUhhÛh²hubhû)”}”(hhh]”j)”}”(hŒIpInUnknownProtos ”h]”hÌ)”}”(hŒIpInUnknownProtos”h]”hŒIpInUnknownProtos”…””}”(hjxh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K[hjtubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjqh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´K[hhÛh²hubhÌ)”}”(hŒóDefined in `RFC1213 ipInUnknownProtos`_. It will be increased if the layer 4 protocol is unsupported by kernel. If an application is using raw socket, kernel will always deliver the packet to the raw socket and this counter won't be increased.”h]”(hŒ Defined in ”…””}”(hj’h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipInUnknownProtos`_”h]”hŒRFC1213 ipInUnknownProtos”…””}”(hjšh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipInUnknownProtos”j;Œ+https://tools.ietf.org/html/rfc1213#page-27”uh1j)hj’j=KubhŒÎ. It will be increased if the layer 4 protocol is unsupported by kernel. If an application is using raw socket, kernel will always deliver the packet to the raw socket and this counter won’t be increased.”…””}”(hj’h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K]hhÛh²hubjE)”}”(hŒJ.. _RFC1213 ipInUnknownProtos: https://tools.ietf.org/html/rfc1213#page-27”h]”h}”(h]”Œrfc1213-ipinunknownprotos”ah ]”h"]”Œrfc1213 ipinunknownprotos”ah$]”h&]”j;jªuh1jDh´KbhhÛh²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒIpExtInTruncatedPkts ”h]”hÌ)”}”(hŒIpExtInTruncatedPkts”h]”hŒIpExtInTruncatedPkts”…””}”(hjÈh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KdhjÄubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjÁh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KdhhÛh²hubhÌ)”}”(hŒkFor IPv4 packet, it means the actual data size is smaller than the "Total Length" field in the IPv4 header.”h]”hŒoFor IPv4 packet, it means the actual data size is smaller than the “Total Length†field in the IPv4 header.”…””}”(hjâh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KfhhÛh²hubhû)”}”(hhh]”j)”}”(hŒ IpInDiscards ”h]”hÌ)”}”(hŒ IpInDiscards”h]”hŒ IpInDiscards”…””}”(hj÷h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kihjóubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjðh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KihhÛh²hubhÌ)”}”(hŒ›Defined in `RFC1213 ipInDiscards`_. It indicates the packet is dropped in the IP receiving path and due to kernel internal reasons (e.g. no enough memory).”h]”(hŒ Defined in ”…””}”(hjh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipInDiscards`_”h]”hŒRFC1213 ipInDiscards”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipInDiscards”j;Œ+https://tools.ietf.org/html/rfc1213#page-28”uh1j)hjj=KubhŒy. It indicates the packet is dropped in the IP receiving path and due to kernel internal reasons (e.g. no enough memory).”…””}”(hjh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KkhhÛh²hubjE)”}”(hŒE.. _RFC1213 ipInDiscards: https://tools.ietf.org/html/rfc1213#page-28”h]”h}”(h]”Œrfc1213-ipindiscards”ah ]”h"]”Œrfc1213 ipindiscards”ah$]”h&]”j;j)uh1jDh´KohhÛh²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒIpOutDiscards ”h]”hÌ)”}”(hŒ IpOutDiscards”h]”hŒ IpOutDiscards”…””}”(hjGh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KqhjCubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj@h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KqhhÛh²hubhÌ)”}”(hŒ‚Defined in `RFC1213 ipOutDiscards`_. It indicates the packet is dropped in the IP sending path and due to kernel internal reasons.”h]”(hŒ Defined in ”…””}”(hjah²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipOutDiscards`_”h]”hŒRFC1213 ipOutDiscards”…””}”(hjih²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipOutDiscards”j;Œ+https://tools.ietf.org/html/rfc1213#page-28”uh1j)hjaj=KubhŒ_. It indicates the packet is dropped in the IP sending path and due to kernel internal reasons.”…””}”(hjah²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KshhÛh²hubjE)”}”(hŒF.. _RFC1213 ipOutDiscards: https://tools.ietf.org/html/rfc1213#page-28”h]”h}”(h]”Œrfc1213-ipoutdiscards”ah ]”h"]”Œrfc1213 ipoutdiscards”ah$]”h&]”j;jyuh1jDh´KvhhÛh²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒIpOutNoRoutes ”h]”hÌ)”}”(hŒ IpOutNoRoutes”h]”hŒ IpOutNoRoutes”…””}”(hj—h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kxhj“ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KxhhÛh²hubhÌ)”}”(hŒ|Defined in `RFC1213 ipOutNoRoutes`_. It indicates the packet is dropped in the IP sending path and no route is found for it.”h]”(hŒ Defined in ”…””}”(hj±h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 ipOutNoRoutes`_”h]”hŒRFC1213 ipOutNoRoutes”…””}”(hj¹h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 ipOutNoRoutes”j;Œ+https://tools.ietf.org/html/rfc1213#page-29”uh1j)hj±j=KubhŒY. It indicates the packet is dropped in the IP sending path and no route is found for it.”…””}”(hj±h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KzhhÛh²hubjE)”}”(hŒF.. _RFC1213 ipOutNoRoutes: https://tools.ietf.org/html/rfc1213#page-29”h]”h}”(h]”Œrfc1213-ipoutnoroutes”ah ]”h"]”Œrfc1213 ipoutnoroutes”ah$]”h&]”j;jÉuh1jDh´K}hhÛh²hh³hÊjRKubeh}”(h]”Œgeneral-ipv4-counters”ah ]”h"]”Œgeneral ipv4 counters”ah$]”h&]”uh1hµhh·h²hh³hÊh´Kubh¶)”}”(hhh]”(h»)”}”(hŒ ICMP counters”h]”hŒ ICMP counters”…””}”(hjëh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjèh²hh³hÊh´K€ubhû)”}”(hhh]”j)”}”(hŒIcmpInMsgs and IcmpOutMsgs ”h]”hÌ)”}”(hŒIcmpInMsgs and IcmpOutMsgs”h]”hŒIcmpInMsgs and IcmpOutMsgs”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khjüubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjùh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Khjèh²hubhÌ)”}”(hŒ;Defined by `RFC1213 icmpInMsgs`_ and `RFC1213 icmpOutMsgs`_”h]”(hŒ Defined by ”…””}”(hjh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInMsgs`_”h]”hŒRFC1213 icmpInMsgs”…””}”(hj"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInMsgs”j;Œ+https://tools.ietf.org/html/rfc1213#page-41”uh1j)hjj=KubhŒ and ”…””}”(hjh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutMsgs`_”h]”hŒRFC1213 icmpOutMsgs”…””}”(hj7h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutMsgs”j;Œ+https://tools.ietf.org/html/rfc1213#page-43”uh1j)hjj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kƒhjèh²hubjE)”}”(hŒC.. _RFC1213 icmpInMsgs: https://tools.ietf.org/html/rfc1213#page-41”h]”h}”(h]”Œrfc1213-icmpinmsgs”ah ]”h"]”Œrfc1213 icmpinmsgs”ah$]”h&]”j;j2uh1jDh´K…hjèh²hh³hÊjRKubjE)”}”(hŒD.. _RFC1213 icmpOutMsgs: https://tools.ietf.org/html/rfc1213#page-43”h]”h}”(h]”Œrfc1213-icmpoutmsgs”ah ]”h"]”Œrfc1213 icmpoutmsgs”ah$]”h&]”j;jGuh1jDh´K†hjèh²hh³hÊjRKubhÌ)”}”(hX"As mentioned in the RFC1213, these two counters include errors, they would be increased even if the ICMP packet has an invalid type. The ICMP output path will check the header of a raw socket, so the IcmpOutMsgs would still be updated if the IP header is constructed by a userspace program.”h]”hX"As mentioned in the RFC1213, these two counters include errors, they would be increased even if the ICMP packet has an invalid type. The ICMP output path will check the header of a raw socket, so the IcmpOutMsgs would still be updated if the IP header is constructed by a userspace program.”…””}”(hjfh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kˆhjèh²hubhû)”}”(hhh]”j)”}”(hŒICMP named types ”h]”hÌ)”}”(hŒICMP named types”h]”hŒICMP named types”…””}”(hj{h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KŽhjwubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjth²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KŽhjèh²hubhŒ line_block”“”)”}”(hhh]”(hh´“”)”}”(hŒ;These counters include most of common ICMP types, they are:”h]”hŒ;These counters include most of common ICMP types, they are:”…””}”(hj›h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h´Œindent”Khj—h²hh³hÊh´Kubjš)”}”(hŒ1IcmpInDestUnreachs: `RFC1213 icmpInDestUnreachs`_”h]”(hŒIcmpInDestUnreachs: ”…””}”(hjªh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInDestUnreachs`_”h]”hŒRFC1213 icmpInDestUnreachs”…””}”(hj²h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInDestUnreachs”j;Œ+https://tools.ietf.org/html/rfc1213#page-41”uh1j)hjªj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K‘ubjš)”}”(hŒ+IcmpInTimeExcds: `RFC1213 icmpInTimeExcds`_”h]”(hŒIcmpInTimeExcds: ”…””}”(hjÉh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInTimeExcds`_”h]”hŒRFC1213 icmpInTimeExcds”…””}”(hjÑh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInTimeExcds”j;Œ+https://tools.ietf.org/html/rfc1213#page-41”uh1j)hjÉj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K’ubjš)”}”(hŒ+IcmpInParmProbs: `RFC1213 icmpInParmProbs`_”h]”(hŒIcmpInParmProbs: ”…””}”(hjèh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInParmProbs`_”h]”hŒRFC1213 icmpInParmProbs”…””}”(hjðh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInParmProbs”j;Œ+https://tools.ietf.org/html/rfc1213#page-42”uh1j)hjèj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K“ubjš)”}”(hŒ-IcmpInSrcQuenchs: `RFC1213 icmpInSrcQuenchs`_”h]”(hŒIcmpInSrcQuenchs: ”…””}”(hjh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInSrcQuenchs`_”h]”hŒRFC1213 icmpInSrcQuenchs”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInSrcQuenchs”j;Œ+https://tools.ietf.org/html/rfc1213#page-42”uh1j)hjj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K”ubjš)”}”(hŒ+IcmpInRedirects: `RFC1213 icmpInRedirects`_”h]”(hŒIcmpInRedirects: ”…””}”(hj&h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInRedirects`_”h]”hŒRFC1213 icmpInRedirects”…””}”(hj.h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInRedirects”j;Œ+https://tools.ietf.org/html/rfc1213#page-42”uh1j)hj&j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K•ubjš)”}”(hŒ#IcmpInEchos: `RFC1213 icmpInEchos`_”h]”(hŒ IcmpInEchos: ”…””}”(hjEh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInEchos`_”h]”hŒRFC1213 icmpInEchos”…””}”(hjMh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInEchos”j;Œ+https://tools.ietf.org/html/rfc1213#page-42”uh1j)hjEj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K–ubjš)”}”(hŒ)IcmpInEchoReps: `RFC1213 icmpInEchoReps`_”h]”(hŒIcmpInEchoReps: ”…””}”(hjdh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInEchoReps`_”h]”hŒRFC1213 icmpInEchoReps”…””}”(hjlh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInEchoReps”j;Œ+https://tools.ietf.org/html/rfc1213#page-42”uh1j)hjdj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K—ubjš)”}”(hŒ-IcmpInTimestamps: `RFC1213 icmpInTimestamps`_”h]”(hŒIcmpInTimestamps: ”…””}”(hjƒh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInTimestamps`_”h]”hŒRFC1213 icmpInTimestamps”…””}”(hj‹h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInTimestamps”j;Œ+https://tools.ietf.org/html/rfc1213#page-42”uh1j)hjƒj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K˜ubjš)”}”(hŒ3IcmpInTimestampReps: `RFC1213 icmpInTimestampReps`_”h]”(hŒIcmpInTimestampReps: ”…””}”(hj¢h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInTimestampReps`_”h]”hŒRFC1213 icmpInTimestampReps”…””}”(hjªh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInTimestampReps”j;Œ+https://tools.ietf.org/html/rfc1213#page-43”uh1j)hj¢j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K™ubjš)”}”(hŒ+IcmpInAddrMasks: `RFC1213 icmpInAddrMasks`_”h]”(hŒIcmpInAddrMasks: ”…””}”(hjÁh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInAddrMasks`_”h]”hŒRFC1213 icmpInAddrMasks”…””}”(hjÉh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInAddrMasks”j;Œ+https://tools.ietf.org/html/rfc1213#page-43”uh1j)hjÁj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´Kšubjš)”}”(hŒ1IcmpInAddrMaskReps: `RFC1213 icmpInAddrMaskReps`_”h]”(hŒIcmpInAddrMaskReps: ”…””}”(hjàh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInAddrMaskReps`_”h]”hŒRFC1213 icmpInAddrMaskReps”…””}”(hjèh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInAddrMaskReps”j;Œ+https://tools.ietf.org/html/rfc1213#page-43”uh1j)hjàj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K›ubjš)”}”(hŒ3IcmpOutDestUnreachs: `RFC1213 icmpOutDestUnreachs`_”h]”(hŒIcmpOutDestUnreachs: ”…””}”(hjÿh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutDestUnreachs`_”h]”hŒRFC1213 icmpOutDestUnreachs”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutDestUnreachs”j;Œ+https://tools.ietf.org/html/rfc1213#page-44”uh1j)hjÿj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´Kœubjš)”}”(hŒ-IcmpOutTimeExcds: `RFC1213 icmpOutTimeExcds`_”h]”(hŒIcmpOutTimeExcds: ”…””}”(hjh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutTimeExcds`_”h]”hŒRFC1213 icmpOutTimeExcds”…””}”(hj&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutTimeExcds”j;Œ+https://tools.ietf.org/html/rfc1213#page-44”uh1j)hjj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´Kubjš)”}”(hŒ-IcmpOutParmProbs: `RFC1213 icmpOutParmProbs`_”h]”(hŒIcmpOutParmProbs: ”…””}”(hj=h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutParmProbs`_”h]”hŒRFC1213 icmpOutParmProbs”…””}”(hjEh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutParmProbs”j;Œ+https://tools.ietf.org/html/rfc1213#page-44”uh1j)hj=j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´Kžubjš)”}”(hŒ/IcmpOutSrcQuenchs: `RFC1213 icmpOutSrcQuenchs`_”h]”(hŒIcmpOutSrcQuenchs: ”…””}”(hj\h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutSrcQuenchs`_”h]”hŒRFC1213 icmpOutSrcQuenchs”…””}”(hjdh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutSrcQuenchs”j;Œ+https://tools.ietf.org/html/rfc1213#page-44”uh1j)hj\j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´KŸubjš)”}”(hŒ-IcmpOutRedirects: `RFC1213 icmpOutRedirects`_”h]”(hŒIcmpOutRedirects: ”…””}”(hj{h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutRedirects`_”h]”hŒRFC1213 icmpOutRedirects”…””}”(hjƒh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutRedirects”j;Œ+https://tools.ietf.org/html/rfc1213#page-44”uh1j)hj{j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K ubjš)”}”(hŒ%IcmpOutEchos: `RFC1213 icmpOutEchos`_”h]”(hŒIcmpOutEchos: ”…””}”(hjšh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutEchos`_”h]”hŒRFC1213 icmpOutEchos”…””}”(hj¢h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutEchos”j;Œ+https://tools.ietf.org/html/rfc1213#page-45”uh1j)hjšj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K¡ubjš)”}”(hŒ+IcmpOutEchoReps: `RFC1213 icmpOutEchoReps`_”h]”(hŒIcmpOutEchoReps: ”…””}”(hj¹h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutEchoReps`_”h]”hŒRFC1213 icmpOutEchoReps”…””}”(hjÁh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutEchoReps”j;Œ+https://tools.ietf.org/html/rfc1213#page-45”uh1j)hj¹j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K¢ubjš)”}”(hŒ/IcmpOutTimestamps: `RFC1213 icmpOutTimestamps`_”h]”(hŒIcmpOutTimestamps: ”…””}”(hjØh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutTimestamps`_”h]”hŒRFC1213 icmpOutTimestamps”…””}”(hjàh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutTimestamps”j;Œ+https://tools.ietf.org/html/rfc1213#page-45”uh1j)hjØj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K£ubjš)”}”(hŒ5IcmpOutTimestampReps: `RFC1213 icmpOutTimestampReps`_”h]”(hŒIcmpOutTimestampReps: ”…””}”(hj÷h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutTimestampReps`_”h]”hŒRFC1213 icmpOutTimestampReps”…””}”(hjÿh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutTimestampReps”j;Œ+https://tools.ietf.org/html/rfc1213#page-45”uh1j)hj÷j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K¤ubjš)”}”(hŒ-IcmpOutAddrMasks: `RFC1213 icmpOutAddrMasks`_”h]”(hŒIcmpOutAddrMasks: ”…””}”(hjh²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutAddrMasks`_”h]”hŒRFC1213 icmpOutAddrMasks”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutAddrMasks”j;Œ+https://tools.ietf.org/html/rfc1213#page-45”uh1j)hjj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K¥ubjš)”}”(hŒ3IcmpOutAddrMaskReps: `RFC1213 icmpOutAddrMaskReps`_”h]”(hŒIcmpOutAddrMaskReps: ”…””}”(hj5h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutAddrMaskReps`_”h]”hŒRFC1213 icmpOutAddrMaskReps”…””}”(hj=h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutAddrMaskReps”j;Œ+https://tools.ietf.org/html/rfc1213#page-46”uh1j)hj5j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1h´j©Khj—h²hh³hÊh´K¦ubeh}”(h]”h ]”h"]”h$]”h&]”uh1j•hjèh²hh³hÊh´KubjE)”}”(hŒK.. _RFC1213 icmpInDestUnreachs: https://tools.ietf.org/html/rfc1213#page-41”h]”h}”(h]”Œrfc1213-icmpindestunreachs”ah ]”h"]”Œrfc1213 icmpindestunreachs”ah$]”h&]”j;jÂuh1jDh´K¨hjèh²hh³hÊjRKubjE)”}”(hŒH.. _RFC1213 icmpInTimeExcds: https://tools.ietf.org/html/rfc1213#page-41”h]”h}”(h]”Œrfc1213-icmpintimeexcds”ah ]”h"]”Œrfc1213 icmpintimeexcds”ah$]”h&]”j;jáuh1jDh´K©hjèh²hh³hÊjRKubjE)”}”(hŒH.. _RFC1213 icmpInParmProbs: https://tools.ietf.org/html/rfc1213#page-42”h]”h}”(h]”Œrfc1213-icmpinparmprobs”ah ]”h"]”Œrfc1213 icmpinparmprobs”ah$]”h&]”j;juh1jDh´Kªhjèh²hh³hÊjRKubjE)”}”(hŒI.. _RFC1213 icmpInSrcQuenchs: https://tools.ietf.org/html/rfc1213#page-42”h]”h}”(h]”Œrfc1213-icmpinsrcquenchs”ah ]”h"]”Œrfc1213 icmpinsrcquenchs”ah$]”h&]”j;juh1jDh´K«hjèh²hh³hÊjRKubjE)”}”(hŒH.. _RFC1213 icmpInRedirects: https://tools.ietf.org/html/rfc1213#page-42”h]”h}”(h]”Œrfc1213-icmpinredirects”ah ]”h"]”Œrfc1213 icmpinredirects”ah$]”h&]”j;j>uh1jDh´K¬hjèh²hh³hÊjRKubjE)”}”(hŒD.. _RFC1213 icmpInEchos: https://tools.ietf.org/html/rfc1213#page-42”h]”h}”(h]”Œrfc1213-icmpinechos”ah ]”h"]”Œrfc1213 icmpinechos”ah$]”h&]”j;j]uh1jDh´K­hjèh²hh³hÊjRKubjE)”}”(hŒG.. _RFC1213 icmpInEchoReps: https://tools.ietf.org/html/rfc1213#page-42”h]”h}”(h]”Œrfc1213-icmpinechoreps”ah ]”h"]”Œrfc1213 icmpinechoreps”ah$]”h&]”j;j|uh1jDh´K®hjèh²hh³hÊjRKubjE)”}”(hŒI.. _RFC1213 icmpInTimestamps: https://tools.ietf.org/html/rfc1213#page-42”h]”h}”(h]”Œrfc1213-icmpintimestamps”ah ]”h"]”Œrfc1213 icmpintimestamps”ah$]”h&]”j;j›uh1jDh´K¯hjèh²hh³hÊjRKubjE)”}”(hŒL.. _RFC1213 icmpInTimestampReps: https://tools.ietf.org/html/rfc1213#page-43”h]”h}”(h]”Œrfc1213-icmpintimestampreps”ah ]”h"]”Œrfc1213 icmpintimestampreps”ah$]”h&]”j;jºuh1jDh´K°hjèh²hh³hÊjRKubjE)”}”(hŒH.. _RFC1213 icmpInAddrMasks: https://tools.ietf.org/html/rfc1213#page-43”h]”h}”(h]”Œrfc1213-icmpinaddrmasks”ah ]”h"]”Œrfc1213 icmpinaddrmasks”ah$]”h&]”j;jÙuh1jDh´K±hjèh²hh³hÊjRKubjE)”}”(hŒK.. _RFC1213 icmpInAddrMaskReps: https://tools.ietf.org/html/rfc1213#page-43”h]”h}”(h]”Œrfc1213-icmpinaddrmaskreps”ah ]”h"]”Œrfc1213 icmpinaddrmaskreps”ah$]”h&]”j;jøuh1jDh´K²hjèh²hh³hÊjRKubjE)”}”(hŒL.. _RFC1213 icmpOutDestUnreachs: https://tools.ietf.org/html/rfc1213#page-44”h]”h}”(h]”Œrfc1213-icmpoutdestunreachs”ah ]”h"]”Œrfc1213 icmpoutdestunreachs”ah$]”h&]”j;juh1jDh´K´hjèh²hh³hÊjRKubjE)”}”(hŒI.. _RFC1213 icmpOutTimeExcds: https://tools.ietf.org/html/rfc1213#page-44”h]”h}”(h]”Œrfc1213-icmpouttimeexcds”ah ]”h"]”Œrfc1213 icmpouttimeexcds”ah$]”h&]”j;j6uh1jDh´Kµhjèh²hh³hÊjRKubjE)”}”(hŒI.. _RFC1213 icmpOutParmProbs: https://tools.ietf.org/html/rfc1213#page-44”h]”h}”(h]”Œrfc1213-icmpoutparmprobs”ah ]”h"]”Œrfc1213 icmpoutparmprobs”ah$]”h&]”j;jUuh1jDh´K¶hjèh²hh³hÊjRKubjE)”}”(hŒJ.. _RFC1213 icmpOutSrcQuenchs: https://tools.ietf.org/html/rfc1213#page-44”h]”h}”(h]”Œrfc1213-icmpoutsrcquenchs”ah ]”h"]”Œrfc1213 icmpoutsrcquenchs”ah$]”h&]”j;jtuh1jDh´K·hjèh²hh³hÊjRKubjE)”}”(hŒI.. _RFC1213 icmpOutRedirects: https://tools.ietf.org/html/rfc1213#page-44”h]”h}”(h]”Œrfc1213-icmpoutredirects”ah ]”h"]”Œrfc1213 icmpoutredirects”ah$]”h&]”j;j“uh1jDh´K¸hjèh²hh³hÊjRKubjE)”}”(hŒE.. _RFC1213 icmpOutEchos: https://tools.ietf.org/html/rfc1213#page-45”h]”h}”(h]”Œrfc1213-icmpoutechos”ah ]”h"]”Œrfc1213 icmpoutechos”ah$]”h&]”j;j²uh1jDh´K¹hjèh²hh³hÊjRKubjE)”}”(hŒH.. _RFC1213 icmpOutEchoReps: https://tools.ietf.org/html/rfc1213#page-45”h]”h}”(h]”Œrfc1213-icmpoutechoreps”ah ]”h"]”Œrfc1213 icmpoutechoreps”ah$]”h&]”j;jÑuh1jDh´Kºhjèh²hh³hÊjRKubjE)”}”(hŒJ.. _RFC1213 icmpOutTimestamps: https://tools.ietf.org/html/rfc1213#page-45”h]”h}”(h]”Œrfc1213-icmpouttimestamps”ah ]”h"]”Œrfc1213 icmpouttimestamps”ah$]”h&]”j;jðuh1jDh´K»hjèh²hh³hÊjRKubjE)”}”(hŒM.. _RFC1213 icmpOutTimestampReps: https://tools.ietf.org/html/rfc1213#page-45”h]”h}”(h]”Œrfc1213-icmpouttimestampreps”ah ]”h"]”Œrfc1213 icmpouttimestampreps”ah$]”h&]”j;juh1jDh´K¼hjèh²hh³hÊjRKubjE)”}”(hŒI.. _RFC1213 icmpOutAddrMasks: https://tools.ietf.org/html/rfc1213#page-45”h]”h}”(h]”Œrfc1213-icmpoutaddrmasks”ah ]”h"]”Œrfc1213 icmpoutaddrmasks”ah$]”h&]”j;j.uh1jDh´K½hjèh²hh³hÊjRKubjE)”}”(hŒL.. _RFC1213 icmpOutAddrMaskReps: https://tools.ietf.org/html/rfc1213#page-46”h]”h}”(h]”Œrfc1213-icmpoutaddrmaskreps”ah ]”h"]”Œrfc1213 icmpoutaddrmaskreps”ah$]”h&]”j;jMuh1jDh´K¾hjèh²hh³hÊjRKubhÌ)”}”(hX Every ICMP type has two counters: 'In' and 'Out'. E.g., for the ICMP Echo packet, they are IcmpInEchos and IcmpOutEchos. Their meanings are straightforward. The 'In' counter means kernel receives such a packet and the 'Out' counter means kernel sends such a packet.”h]”hXEvery ICMP type has two counters: ‘In’ and ‘Out’. E.g., for the ICMP Echo packet, they are IcmpInEchos and IcmpOutEchos. Their meanings are straightforward. The ‘In’ counter means kernel receives such a packet and the ‘Out’ counter means kernel sends such a packet.”…””}”(hjb h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÀhjèh²hubhû)”}”(hhh]”j)”}”(hŒICMP numeric types ”h]”hÌ)”}”(hŒICMP numeric types”h]”hŒICMP numeric types”…””}”(hjw h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÅhjs ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjp h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KÅhjèh²hubhÌ)”}”(hŒÛThey are IcmpMsgInType[N] and IcmpMsgOutType[N], the [N] indicates the ICMP type number. These counters track all kinds of ICMP packets. The ICMP type number definition could be found in the `ICMP parameters`_ document.”h]”(hŒ¿They are IcmpMsgInType[N] and IcmpMsgOutType[N], the [N] indicates the ICMP type number. These counters track all kinds of ICMP packets. The ICMP type number definition could be found in the ”…””}”(hj‘ h²hh³Nh´Nubj*)”}”(hŒ`ICMP parameters`_”h]”hŒICMP parameters”…””}”(hj™ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒICMP parameters”j;ŒFhttps://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml”uh1j)hj‘ j=KubhŒ document.”…””}”(hj‘ h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÇhjèh²hubjE)”}”(hŒ[.. _ICMP parameters: https://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml”h]”h}”(h]”Œicmp-parameters”ah ]”h"]”Œicmp parameters”ah$]”h&]”j;j© uh1jDh´KÌhjèh²hh³hÊjRKubhÌ)”}”(hŒ°For example, if the Linux kernel sends an ICMP Echo packet, the IcmpMsgOutType8 would increase 1. And if kernel gets an ICMP Echo Reply packet, IcmpMsgInType0 would increase 1.”h]”hŒ°For example, if the Linux kernel sends an ICMP Echo packet, the IcmpMsgOutType8 would increase 1. And if kernel gets an ICMP Echo Reply packet, IcmpMsgInType0 would increase 1.”…””}”(hjÀ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÎhjèh²hubhû)”}”(hhh]”j)”}”(hŒIcmpInCsumErrors ”h]”hÌ)”}”(hŒIcmpInCsumErrors”h]”hŒIcmpInCsumErrors”…””}”(hjÕ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÒhjÑ ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjÎ h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KÒhjèh²hubhÌ)”}”(hXThis counter indicates the checksum of the ICMP packet is wrong. Kernel verifies the checksum after updating the IcmpInMsgs and before updating IcmpMsgInType[N]. If a packet has bad checksum, the IcmpInMsgs would be updated but none of IcmpMsgInType[N] would be updated.”h]”hXThis counter indicates the checksum of the ICMP packet is wrong. Kernel verifies the checksum after updating the IcmpInMsgs and before updating IcmpMsgInType[N]. If a packet has bad checksum, the IcmpInMsgs would be updated but none of IcmpMsgInType[N] would be updated.”…””}”(hjï h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÔhjèh²hubhû)”}”(hhh]”j)”}”(hŒIcmpInErrors and IcmpOutErrors ”h]”hÌ)”}”(hŒIcmpInErrors and IcmpOutErrors”h]”hŒIcmpInErrors and IcmpOutErrors”…””}”(hj h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÙhj ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjý h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´KÙhjèh²hubhÌ)”}”(hŒ?Defined by `RFC1213 icmpInErrors`_ and `RFC1213 icmpOutErrors`_”h]”(hŒ Defined by ”…””}”(hj h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpInErrors`_”h]”hŒRFC1213 icmpInErrors”…””}”(hj& h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpInErrors”j;Œ+https://tools.ietf.org/html/rfc1213#page-41”uh1j)hj j=KubhŒ and ”…””}”(hj h²hh³Nh´Nubj*)”}”(hŒ`RFC1213 icmpOutErrors`_”h]”hŒRFC1213 icmpOutErrors”…””}”(hj; h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC1213 icmpOutErrors”j;Œ+https://tools.ietf.org/html/rfc1213#page-43”uh1j)hj j=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÛhjèh²hubjE)”}”(hŒE.. _RFC1213 icmpInErrors: https://tools.ietf.org/html/rfc1213#page-41”h]”h}”(h]”Œrfc1213-icmpinerrors”ah ]”h"]”Œrfc1213 icmpinerrors”ah$]”h&]”j;j6 uh1jDh´KÝhjèh²hh³hÊjRKubjE)”}”(hŒF.. _RFC1213 icmpOutErrors: https://tools.ietf.org/html/rfc1213#page-43”h]”h}”(h]”Œrfc1213-icmpouterrors”ah ]”h"]”Œrfc1213 icmpouterrors”ah$]”h&]”j;jK uh1jDh´KÞhjèh²hh³hÊjRKubhÌ)”}”(hXWhen an error occurs in the ICMP packet handler path, these two counters would be updated. The receiving packet path use IcmpInErrors and the sending packet path use IcmpOutErrors. When IcmpInCsumErrors is increased, IcmpInErrors would always be increased too.”h]”hXWhen an error occurs in the ICMP packet handler path, these two counters would be updated. The receiving packet path use IcmpInErrors and the sending packet path use IcmpOutErrors. When IcmpInCsumErrors is increased, IcmpInErrors would always be increased too.”…””}”(hjj h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kàhjèh²hubh¶)”}”(hhh]”(h»)”}”(hŒ!relationship of the ICMP counters”h]”hŒ!relationship of the ICMP counters”…””}”(hj{ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjx h²hh³hÊh´KæubhÌ)”}”(hŒüThe sum of IcmpMsgOutType[N] is always equal to IcmpOutMsgs, as they are updated at the same time. The sum of IcmpMsgInType[N] plus IcmpInErrors should be equal or larger than IcmpInMsgs. When kernel receives an ICMP packet, kernel follows below logic:”h]”hŒüThe sum of IcmpMsgOutType[N] is always equal to IcmpOutMsgs, as they are updated at the same time. The sum of IcmpMsgInType[N] plus IcmpInErrors should be equal or larger than IcmpInMsgs. When kernel receives an ICMP packet, kernel follows below logic:”…””}”(hj‰ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kçhjx h²hubhŒenumerated_list”“”)”}”(hhh]”(j)”}”(hŒincrease IcmpInMsgs”h]”hÌ)”}”(hjž h]”hŒincrease IcmpInMsgs”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kìhjœ ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj™ h²hh³hÊh´Nubj)”}”(hŒPackets are dropped by PAWS in any status other than Syn-Sent.”h]”hŒ>Packets are dropped by PAWS in any status other than Syn-Sent.”…””}”(hjWh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M hj¦h²hubeh}”(h]”Œtcp-paws”ah ]”h"]”Œtcp paws”ah$]”h&]”uh1hµhh·h²hh³hÊh´Müubh¶)”}”(hhh]”(h»)”}”(hŒ TCP ACK skip”h]”hŒ TCP ACK skip”…””}”(hjph²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjmh²hh³hÊh´MubhÌ)”}”(hX¡In some scenarios, kernel would avoid sending duplicate ACKs too frequently. Please find more details in the tcp_invalid_ratelimit section of the `sysctl document`_. When kernel decides to skip an ACK due to tcp_invalid_ratelimit, kernel would update one of below counters to indicate the ACK is skipped in which scenario. The ACK would only be skipped if the received packet is either a SYN packet or it has no data.”h]”(hŒ’In some scenarios, kernel would avoid sending duplicate ACKs too frequently. Please find more details in the tcp_invalid_ratelimit section of the ”…””}”(hj~h²hh³Nh´Nubj*)”}”(hŒ`sysctl document`_”h]”hŒsysctl document”…””}”(hj†h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”Œsysctl document”j;ŒAhttps://www.kernel.org/doc/Documentation/networking/ip-sysctl.rst”uh1j)hj~j=KubhŒý. When kernel decides to skip an ACK due to tcp_invalid_ratelimit, kernel would update one of below counters to indicate the ACK is skipped in which scenario. The ACK would only be skipped if the received packet is either a SYN packet or it has no data.”…””}”(hj~h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhjmh²hubjE)”}”(hŒV.. _sysctl document: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.rst”h]”h}”(h]”Œsysctl-document”ah ]”h"]”Œsysctl document”ah$]”h&]”j;j–uh1jDh´Mhjmh²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPACKSkippedSynRecv ”h]”hÌ)”}”(hŒTcpExtTCPACKSkippedSynRecv”h]”hŒTcpExtTCPACKSkippedSynRecv”…””}”(hj´h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhj°ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj­h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mhjmh²hubhÌ)”}”(hX•The ACK is skipped in Syn-Recv status. The Syn-Recv status means the TCP stack receives a SYN and replies SYN+ACK. Now the TCP stack is waiting for an ACK. Generally, the TCP stack doesn't need to send ACK in the Syn-Recv status. But in several scenarios, the TCP stack need to send an ACK. E.g., the TCP stack receives the same SYN packet repeately, the received packet does not pass the PAWS check, or the received packet sequence number is out of window. In these scenarios, the TCP stack needs to send ACK. If the ACk sending frequency is higher than tcp_invalid_ratelimit allows, the TCP stack will skip sending ACK and increase TcpExtTCPACKSkippedSynRecv.”h]”hX—The ACK is skipped in Syn-Recv status. The Syn-Recv status means the TCP stack receives a SYN and replies SYN+ACK. Now the TCP stack is waiting for an ACK. Generally, the TCP stack doesn’t need to send ACK in the Syn-Recv status. But in several scenarios, the TCP stack need to send an ACK. E.g., the TCP stack receives the same SYN packet repeately, the received packet does not pass the PAWS check, or the received packet sequence number is out of window. In these scenarios, the TCP stack needs to send ACK. If the ACk sending frequency is higher than tcp_invalid_ratelimit allows, the TCP stack will skip sending ACK and increase TcpExtTCPACKSkippedSynRecv.”…””}”(hjÎh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhjmh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPACKSkippedPAWS ”h]”hÌ)”}”(hŒTcpExtTCPACKSkippedPAWS”h]”hŒTcpExtTCPACKSkippedPAWS”…””}”(hjãh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M'hjßubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjÜh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M'hjmh²hubhÌ)”}”(hXmThe ACK is skipped due to PAWS (Protect Against Wrapped Sequence numbers) check fails. If the PAWS check fails in Syn-Recv, Fin-Wait-2 or Time-Wait statuses, the skipped ACK would be counted to TcpExtTCPACKSkippedSynRecv, TcpExtTCPACKSkippedFinWait2 or TcpExtTCPACKSkippedTimeWait. In all other statuses, the skipped ACK would be counted to TcpExtTCPACKSkippedPAWS.”h]”hXmThe ACK is skipped due to PAWS (Protect Against Wrapped Sequence numbers) check fails. If the PAWS check fails in Syn-Recv, Fin-Wait-2 or Time-Wait statuses, the skipped ACK would be counted to TcpExtTCPACKSkippedSynRecv, TcpExtTCPACKSkippedFinWait2 or TcpExtTCPACKSkippedTimeWait. In all other statuses, the skipped ACK would be counted to TcpExtTCPACKSkippedPAWS.”…””}”(hjýh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M)hjmh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPACKSkippedSeq ”h]”hÌ)”}”(hŒTcpExtTCPACKSkippedSeq”h]”hŒTcpExtTCPACKSkippedSeq”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M0hjubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M0hjmh²hubhÌ)”}”(hŒ‹The sequence number is out of window and the timestamp passes the PAWS check and the TCP status is not Syn-Recv, Fin-Wait-2, and Time-Wait.”h]”hŒ‹The sequence number is out of window and the timestamp passes the PAWS check and the TCP status is not Syn-Recv, Fin-Wait-2, and Time-Wait.”…””}”(hj,h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M2hjmh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPACKSkippedFinWait2 ”h]”hÌ)”}”(hŒTcpExtTCPACKSkippedFinWait2”h]”hŒTcpExtTCPACKSkippedFinWait2”…””}”(hjAh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M5hj=ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj:h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M5hjmh²hubhÌ)”}”(hŒ†The ACK is skipped in Fin-Wait-2 status, the reason would be either PAWS check fails or the received sequence number is out of window.”h]”hŒ†The ACK is skipped in Fin-Wait-2 status, the reason would be either PAWS check fails or the received sequence number is out of window.”…””}”(hj[h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M7hjmh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPACKSkippedTimeWait ”h]”hÌ)”}”(hŒTcpExtTCPACKSkippedTimeWait”h]”hŒTcpExtTCPACKSkippedTimeWait”…””}”(hjph²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M:hjlubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjih²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M:hjmh²hubhÌ)”}”(hŒ†The ACK is skipped in Time-Wait status, the reason would be either PAWS check failed or the received sequence number is out of window.”h]”hŒ†The ACK is skipped in Time-Wait status, the reason would be either PAWS check failed or the received sequence number is out of window.”…””}”(hjŠh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M<hjmh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPACKSkippedChallenge ”h]”hÌ)”}”(hŒTcpExtTCPACKSkippedChallenge”h]”hŒTcpExtTCPACKSkippedChallenge”…””}”(hjŸh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M?hj›ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj˜h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M?hjmh²hubhÌ)”}”(hXˆThe ACK is skipped if the ACK is a challenge ACK. The RFC 5961 defines 3 kind of challenge ACK, please refer `RFC 5961 section 3.2`_, `RFC 5961 section 4.2`_ and `RFC 5961 section 5.2`_. Besides these three scenarios, In some TCP status, the linux TCP stack would also send challenge ACKs if the ACK number is before the first unacknowledged number (more strict than `RFC 5961 section 5.2`_).”h]”(hŒmThe ACK is skipped if the ACK is a challenge ACK. The RFC 5961 defines 3 kind of challenge ACK, please refer ”…””}”(hj¹h²hh³Nh´Nubj*)”}”(hŒ`RFC 5961 section 3.2`_”h]”hŒRFC 5961 section 3.2”…””}”(hjÁh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC 5961 section 3.2”j;Œ*https://tools.ietf.org/html/rfc5961#page-7”uh1j)hj¹j=KubhŒ, ”…””}”(hj¹h²hh³Nh´Nubj*)”}”(hŒ`RFC 5961 section 4.2`_”h]”hŒRFC 5961 section 4.2”…””}”(hjÖh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC 5961 section 4.2”j;Œ*https://tools.ietf.org/html/rfc5961#page-9”uh1j)hj¹j=KubhŒ and ”…””}”(hj¹h²hh³Nh´Nubj*)”}”(hŒ`RFC 5961 section 5.2`_”h]”hŒRFC 5961 section 5.2”…””}”(hjëh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC 5961 section 5.2”j;Œ+https://tools.ietf.org/html/rfc5961#page-11”uh1j)hj¹j=KubhŒ¶. Besides these three scenarios, In some TCP status, the linux TCP stack would also send challenge ACKs if the ACK number is before the first unacknowledged number (more strict than ”…””}”(hj¹h²hh³Nh´Nubj*)”}”(hŒ`RFC 5961 section 5.2`_”h]”hŒRFC 5961 section 5.2”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒRFC 5961 section 5.2”j;jûuh1j)hj¹j=KubhŒ).”…””}”(hj¹h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MAhjmh²hubjE)”}”(hŒD.. _RFC 5961 section 3.2: https://tools.ietf.org/html/rfc5961#page-7”h]”h}”(h]”Œrfc-5961-section-3-2”ah ]”h"]”Œrfc 5961 section 3.2”ah$]”h&]”j;jÑuh1jDh´MHhjmh²hh³hÊjRKubjE)”}”(hŒD.. _RFC 5961 section 4.2: https://tools.ietf.org/html/rfc5961#page-9”h]”h}”(h]”Œrfc-5961-section-4-2”ah ]”h"]”Œrfc 5961 section 4.2”ah$]”h&]”j;jæuh1jDh´MIhjmh²hh³hÊjRKubjE)”}”(hŒE.. _RFC 5961 section 5.2: https://tools.ietf.org/html/rfc5961#page-11”h]”h}”(h]”Œrfc-5961-section-5-2”ah ]”h"]”Œrfc 5961 section 5.2”ah$]”h&]”j;jûuh1jDh´MJhjmh²hh³hÊjRKubeh}”(h]”Œ tcp-ack-skip”ah ]”h"]”Œ tcp ack skip”ah$]”h&]”uh1hµhh·h²hh³hÊh´Mubh¶)”}”(hhh]”(h»)”}”(hŒTCP receive window”h]”hŒTCP receive window”…””}”(hjIh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjFh²hh³hÊh´MMubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPWantZeroWindowAdv ”h]”hÌ)”}”(hŒTcpExtTCPWantZeroWindowAdv”h]”hŒTcpExtTCPWantZeroWindowAdv”…””}”(hj^h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MNhjZubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjWh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´MNhjFh²hubhÌ)”}”(hXCDepending on current memory usage, the TCP stack tries to set receive window to zero. But the receive window might still be a no-zero value. For example, if the previous window size is 10, and the TCP stack receives 3 bytes, the current window size would be 7 even if the window size calculated by the memory usage is zero.”h]”hXCDepending on current memory usage, the TCP stack tries to set receive window to zero. But the receive window might still be a no-zero value. For example, if the previous window size is 10, and the TCP stack receives 3 bytes, the current window size would be 7 even if the window size calculated by the memory usage is zero.”…””}”(hjxh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MPhjFh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPToZeroWindowAdv ”h]”hÌ)”}”(hŒTcpExtTCPToZeroWindowAdv”h]”hŒTcpExtTCPToZeroWindowAdv”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MVhj‰ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj†h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´MVhjFh²hubhÌ)”}”(hŒ;The TCP receive window is set to zero from a no-zero value.”h]”hŒ;The TCP receive window is set to zero from a no-zero value.”…””}”(hj§h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MXhjFh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPFromZeroWindowAdv ”h]”hÌ)”}”(hŒTcpExtTCPFromZeroWindowAdv”h]”hŒTcpExtTCPFromZeroWindowAdv”…””}”(hj¼h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MZhj¸ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjµh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´MZhjFh²hubhÌ)”}”(hŒ9The TCP receive window is set to no-zero value from zero.”h]”hŒ9The TCP receive window is set to no-zero value from zero.”…””}”(hjÖh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M\hjFh²hubeh}”(h]”Œtcp-receive-window”ah ]”h"]”Œtcp receive window”ah$]”h&]”uh1hµhh·h²hh³hÊh´MMubh¶)”}”(hhh]”(h»)”}”(hŒ Delayed ACK”h]”hŒ Delayed ACK”…””}”(hjïh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjìh²hh³hÊh´M`ubhÌ)”}”(hŒ•The TCP Delayed ACK is a technique which is used for reducing the packet count in the network. For more details, please refer the `Delayed ACK wiki`_”h]”(hŒ‚The TCP Delayed ACK is a technique which is used for reducing the packet count in the network. For more details, please refer the ”…””}”(hjýh²hh³Nh´Nubj*)”}”(hŒ`Delayed ACK wiki`_”h]”hŒDelayed ACK wiki”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒDelayed ACK wiki”j;Œ8https://en.wikipedia.org/wiki/TCP_delayed_acknowledgment”uh1j)hjýj=Kubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mahjìh²hubjE)”}”(hŒN.. _Delayed ACK wiki: https://en.wikipedia.org/wiki/TCP_delayed_acknowledgment”h]”h}”(h]”Œdelayed-ack-wiki”ah ]”h"]”Œdelayed ack wiki”ah$]”h&]”j;juh1jDh´Mehjìh²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒTcpExtDelayedACKs ”h]”hÌ)”}”(hŒTcpExtDelayedACKs”h]”hŒTcpExtDelayedACKs”…””}”(hj/h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mghj+ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj(h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mghjìh²hubhÌ)”}”(hŒeA delayed ACK timer expires. The TCP stack will send a pure ACK packet and exit the delayed ACK mode.”h]”hŒeA delayed ACK timer expires. The TCP stack will send a pure ACK packet and exit the delayed ACK mode.”…””}”(hjIh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mihjìh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtDelayedACKLocked ”h]”hÌ)”}”(hŒTcpExtDelayedACKLocked”h]”hŒTcpExtDelayedACKLocked”…””}”(hj^h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MlhjZubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjWh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mlhjìh²hubhÌ)”}”(hXWA delayed ACK timer expires, but the TCP stack can't send an ACK immediately due to the socket is locked by a userspace program. The TCP stack will send a pure ACK later (after the userspace program unlock the socket). When the TCP stack sends the pure ACK later, the TCP stack will also update TcpExtDelayedACKs and exit the delayed ACK mode.”h]”hXYA delayed ACK timer expires, but the TCP stack can’t send an ACK immediately due to the socket is locked by a userspace program. The TCP stack will send a pure ACK later (after the userspace program unlock the socket). When the TCP stack sends the pure ACK later, the TCP stack will also update TcpExtDelayedACKs and exit the delayed ACK mode.”…””}”(hjxh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mnhjìh²hubhû)”}”(hhh]”j)”}”(hŒTcpExtDelayedACKLost ”h]”hÌ)”}”(hŒTcpExtDelayedACKLost”h]”hŒTcpExtDelayedACKLost”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Muhj‰ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj†h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Muhjìh²hubhÌ)”}”(hŒØIt will be updated when the TCP stack receives a packet which has been ACKed. A Delayed ACK loss might cause this issue, but it would also be triggered by other reasons, such as a packet is duplicated in the network.”h]”hŒØIt will be updated when the TCP stack receives a packet which has been ACKed. A Delayed ACK loss might cause this issue, but it would also be triggered by other reasons, such as a packet is duplicated in the network.”…””}”(hj§h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mwhjìh²hubeh}”(h]”Œ delayed-ack”ah ]”h"]”Œ delayed ack”ah$]”h&]”uh1hµhh·h²hh³hÊh´M`ubh¶)”}”(hhh]”(h»)”}”(hŒTail Loss Probe (TLP)”h]”hŒTail Loss Probe (TLP)”…””}”(hjÀh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj½h²hh³hÊh´M}ubhÌ)”}”(hŒmTLP is an algorithm which is used to detect TCP packet loss. For more details, please refer the `TLP paper`_.”h]”(hŒ`TLP is an algorithm which is used to detect TCP packet loss. For more details, please refer the ”…””}”(hjÎh²hh³Nh´Nubj*)”}”(hŒ `TLP paper`_”h]”hŒ TLP paper”…””}”(hjÖh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”Œ TLP paper”j;ŒBhttps://tools.ietf.org/html/draft-dukkipati-tcpm-tcp-loss-probe-01”uh1j)hjÎj=KubhŒ.”…””}”(hjÎh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M~hj½h²hubjE)”}”(hŒQ.. _TLP paper: https://tools.ietf.org/html/draft-dukkipati-tcpm-tcp-loss-probe-01”h]”h}”(h]”Œ tlp-paper”ah ]”h"]”Œ tlp paper”ah$]”h&]”j;jæuh1jDh´Mhj½h²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPLossProbes ”h]”hÌ)”}”(hŒTcpExtTCPLossProbes”h]”hŒTcpExtTCPLossProbes”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mƒhjubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjýh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mƒhj½h²hubhÌ)”}”(hŒA TLP probe packet is sent.”h]”hŒA TLP probe packet is sent.”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M…hj½h²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPLossProbeRecovery ”h]”hÌ)”}”(hŒTcpExtTCPLossProbeRecovery”h]”hŒTcpExtTCPLossProbeRecovery”…””}”(hj3h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M‡hj/ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj,h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M‡hj½h²hubhÌ)”}”(hŒ/A packet loss is detected and recovered by TLP.”h]”hŒ/A packet loss is detected and recovered by TLP.”…””}”(hjMh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M‰hj½h²hubeh}”(h]”Œtail-loss-probe-tlp”ah ]”h"]”Œtail loss probe (tlp)”ah$]”h&]”uh1hµhh·h²hh³hÊh´M}ubh¶)”}”(hhh]”(h»)”}”(hŒTCP Fast Open description”h]”hŒTCP Fast Open description”…””}”(hjfh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjch²hh³hÊh´MŒubhÌ)”}”(hŒŸTCP Fast Open is a technology which allows data transfer before the 3-way handshake complete. Please refer the `TCP Fast Open wiki`_ for a general description.”h]”(hŒoTCP Fast Open is a technology which allows data transfer before the 3-way handshake complete. Please refer the ”…””}”(hjth²hh³Nh´Nubj*)”}”(hŒ`TCP Fast Open wiki`_”h]”hŒTCP Fast Open wiki”…””}”(hj|h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œname”ŒTCP Fast Open wiki”j;Œ+https://en.wikipedia.org/wiki/TCP_Fast_Open”uh1j)hjtj=KubhŒ for a general description.”…””}”(hjth²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhjch²hubjE)”}”(hŒC.. _TCP Fast Open wiki: https://en.wikipedia.org/wiki/TCP_Fast_Open”•—h]”h}”(h]”Œtcp-fast-open-wiki”ah ]”h"]”Œtcp fast open wiki”ah$]”h&]”j;jŒuh1jDh´M‘hjch²hh³hÊjRKubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPFastOpenActive ”h]”hÌ)”}”(hŒTcpExtTCPFastOpenActive”h]”hŒTcpExtTCPFastOpenActive”…””}”(hjªh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M“hj¦ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj£h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M“hjch²hubhÌ)”}”(hŒâWhen the TCP stack receives an ACK packet in the SYN-SENT status, and the ACK packet acknowledges the data in the SYN packet, the TCP stack understand the TFO cookie is accepted by the other side, then it updates this counter.”h]”hŒâWhen the TCP stack receives an ACK packet in the SYN-SENT status, and the ACK packet acknowledges the data in the SYN packet, the TCP stack understand the TFO cookie is accepted by the other side, then it updates this counter.”…””}”(hjÄh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M•hjch²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPFastOpenActiveFail ”h]”hÌ)”}”(hŒTcpExtTCPFastOpenActiveFail”h]”hŒTcpExtTCPFastOpenActiveFail”…””}”(hjÙh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MšhjÕubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjÒh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mšhjch²hubhÌ)”}”(hXµThis counter indicates that the TCP stack initiated a TCP Fast Open, but it failed. This counter would be updated in three scenarios: (1) the other side doesn't acknowledge the data in the SYN packet. (2) The SYN packet which has the TFO cookie is timeout at least once. (3) after the 3-way handshake, the retransmission timeout happens net.ipv4.tcp_retries1 times, because some middle-boxes may black-hole fast open after the handshake.”h]”hX·This counter indicates that the TCP stack initiated a TCP Fast Open, but it failed. This counter would be updated in three scenarios: (1) the other side doesn’t acknowledge the data in the SYN packet. (2) The SYN packet which has the TFO cookie is timeout at least once. (3) after the 3-way handshake, the retransmission timeout happens net.ipv4.tcp_retries1 times, because some middle-boxes may black-hole fast open after the handshake.”…””}”(hjóh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mœhjch²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPFastOpenPassive ”h]”hÌ)”}”(hŒTcpExtTCPFastOpenPassive”h]”hŒTcpExtTCPFastOpenPassive”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¤hjubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjh²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M¤hjch²hubhÌ)”}”(hŒRThis counter indicates how many times the TCP stack accepts the fast open request.”h]”hŒRThis counter indicates how many times the TCP stack accepts the fast open request.”…””}”(hj"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¦hjch²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPFastOpenPassiveFail ”h]”hÌ)”}”(hŒTcpExtTCPFastOpenPassiveFail”h]”hŒTcpExtTCPFastOpenPassiveFail”…””}”(hj7h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M©hj3ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj0h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M©hjch²hubhÌ)”}”(hŒÇThis counter indicates how many times the TCP stack rejects the fast open request. It is caused by either the TFO cookie is invalid or the TCP stack finds an error during the socket creating process.”h]”hŒÇThis counter indicates how many times the TCP stack rejects the fast open request. It is caused by either the TFO cookie is invalid or the TCP stack finds an error during the socket creating process.”…””}”(hjQh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M«hjch²hubhû)”}”(hhh]”j)”}”(hŒ TcpExtTCPFastOpenListenOverflow ”h]”hÌ)”}”(hŒTcpExtTCPFastOpenListenOverflow”h]”hŒTcpExtTCPFastOpenListenOverflow”…””}”(hjfh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¯hjbubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj_h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´M¯hjch²hubhÌ)”}”(hXŽWhen the pending fast open request number is larger than fastopenq->max_qlen, the TCP stack will reject the fast open request and update this counter. When this counter is updated, the TCP stack won't update TcpExtTCPFastOpenPassive or TcpExtTCPFastOpenPassiveFail. The fastopenq->max_qlen is set by the TCP_FASTOPEN socket operation and it could not be larger than net.core.somaxconn. For example:”h]”hXWhen the pending fast open request number is larger than fastopenq->max_qlen, the TCP stack will reject the fast open request and update this counter. When this counter is updated, the TCP stack won’t update TcpExtTCPFastOpenPassive or TcpExtTCPFastOpenPassiveFail. The fastopenq->max_qlen is set by the TCP_FASTOPEN socket operation and it could not be larger than net.core.somaxconn. For example:”…””}”(hj€h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M±hjch²hubhÌ)”}”(hŒ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mçhj- h²hubhû)”}”(hhh]”j)”}”(hŒTcpExtPruneCalled ”h]”hÌ)”}”(hŒTcpExtPruneCalled”h]”hŒTcpExtPruneCalled”…””}”(hjS h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MíhjO ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjL h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Míhj- h²hubhÌ)”}”(hX4The TCP stack tries to reclaim memory for a socket. After updates this counter, the TCP stack will try to collapse the out of order queue and the receiving queue. If the memory is still not enough, the TCP stack will try to discard packets from the out of order queue (and update the TcpExtOfoPruned counter)”h]”hX4The TCP stack tries to reclaim memory for a socket. After updates this counter, the TCP stack will try to collapse the out of order queue and the receiving queue. If the memory is still not enough, the TCP stack will try to discard packets from the out of order queue (and update the TcpExtOfoPruned counter)”…””}”(hjm h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mïhj- h²hubhû)”}”(hhh]”j)”}”(hŒTcpExtOfoPruned ”h]”hÌ)”}”(hŒTcpExtOfoPruned”h]”hŒTcpExtOfoPruned”…””}”(hj‚ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mõhj~ ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhj{ h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mõhj- h²hubhÌ)”}”(hŒ@The TCP stack tries to discard packet on the out of order queue.”h]”hŒ@The TCP stack tries to discard packet on the out of order queue.”…””}”(hjœ h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M÷hj- h²hubhû)”}”(hhh]”j)”}”(hŒTcpExtRcvPruned ”h]”hÌ)”}”(hŒTcpExtRcvPruned”h]”hŒTcpExtRcvPruned”…””}”(hj± h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mùhj­ ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjª h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mùhj- h²hubhÌ)”}”(hŒÄAfter 'collapse' and discard packets from the out of order queue, if the actually used memory is still larger than the max allowed memory, this counter will be updated. It means the 'prune' fails.”h]”hŒÌAfter ‘collapse’ and discard packets from the out of order queue, if the actually used memory is still larger than the max allowed memory, this counter will be updated. It means the ‘prune’ fails.”…””}”(hjË h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mûhj- h²hubhû)”}”(hhh]”j)”}”(hŒTcpExtTCPRcvCollapsed ”h]”hÌ)”}”(hŒTcpExtTCPRcvCollapsed”h]”hŒTcpExtTCPRcvCollapsed”…””}”(hjà h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÿhjÜ ubah}”(h]”h ]”h"]”h$]”h&]”uh1hÿhjÙ h²hh³hÊh´Nubah}”(h]”h ]”h"]”h$]”h&]”jj uh1húh³hÊh´Mÿhj- h²hubhÌ)”}”(hŒAThis counter indicates how many skbs are freed during 'collapse'.”h]”hŒEThis counter indicates how many skbs are freed during ‘collapse’.”…””}”(hjú h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhj- h²hubeh}”(h]”Œprune”ah ]”h"]”Œprune”ah$]”h&]”uh1hµhh·h²hh³hÊh´Mæubh¶)”}”(hhh]”(h»)”}”(hŒexamples”h]”hŒexamples”…””}”(hj!h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj!h²hh³hÊh´Mubh¶)”}”(hhh]”(h»)”}”(hŒ ping test”h]”hŒ ping test”…””}”(hj$!h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj!!h²hh³hÊh´MubhÌ)”}”(hŒhjô!h²hubhÌ)”}”(hŒ0On server side, we can find below nstat output::”h]”hŒ/On server side, we can find below nstat output:”…””}”(hjK"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MAhjô!h²hubj8 )”}”(hXnstatuser@nstat-b:~$ nstat | grep -i tcp TcpPassiveOpens 1 0.0 TcpInSegs 2 0.0 TcpOutSegs 1 0.0 TcpExtTCPPureAcks 1 0.0”h]”hXnstatuser@nstat-b:~$ nstat | grep -i tcp TcpPassiveOpens 1 0.0 TcpInSegs 2 0.0 TcpOutSegs 1 0.0 TcpExtTCPPureAcks 1 0.0”…””}”hjY"sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MChjô!h²hubhÌ)”}”(hŒ0On client side, we can find below nstat output::”h]”hŒ/On client side, we can find below nstat output:”…””}”(hjg"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MIhjô!h²hubj8 )”}”(hŒÍnstatuser@nstat-a:~$ nstat | grep -i tcp TcpActiveOpens 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 2 0.0”h]”hŒÍnstatuser@nstat-a:~$ nstat | grep -i tcp TcpActiveOpens 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 2 0.0”…””}”hju"sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MKhjô!h²hubhÌ)”}”(hXnWhen the server received the first SYN, it replied a SYN+ACK, and came into SYN-RCVD state, so TcpPassiveOpens increased 1. The server received SYN, sent SYN+ACK, received ACK, so server sent 1 packet, received 2 packets, TcpInSegs increased 2, TcpOutSegs increased 1. The last ACK of the 3-way handshake is a pure ACK without data, so TcpExtTCPPureAcks increased 1.”h]”hXnWhen the server received the first SYN, it replied a SYN+ACK, and came into SYN-RCVD state, so TcpPassiveOpens increased 1. The server received SYN, sent SYN+ACK, received ACK, so server sent 1 packet, received 2 packets, TcpInSegs increased 2, TcpOutSegs increased 1. The last ACK of the 3-way handshake is a pure ACK without data, so TcpExtTCPPureAcks increased 1.”…””}”(hjƒ"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MPhjô!h²hubhÌ)”}”(hŒîWhen the client sent SYN, the client came into the SYN-SENT state, so TcpActiveOpens increased 1, the client sent SYN, received SYN+ACK, sent ACK, so client sent 2 packets, received 1 packet, TcpInSegs increased 1, TcpOutSegs increased 2.”h]”hŒîWhen the client sent SYN, the client came into the SYN-SENT state, so TcpActiveOpens increased 1, the client sent SYN, received SYN+ACK, sent ACK, so client sent 2 packets, received 1 packet, TcpInSegs increased 1, TcpOutSegs increased 2.”…””}”(hj‘"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MWhjô!h²hubeh}”(h]”Œtcp-3-way-handshake”ah ]”h"]”Œtcp 3-way handshake”ah$]”h&]”uh1hµhj!h²hh³hÊh´M3ubh¶)”}”(hhh]”(h»)”}”(hŒTCP normal traffic”h]”hŒTCP normal traffic”…””}”(hjª"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj§"h²hh³hÊh´M]ubhÌ)”}”(hŒRun nc on server::”h]”hŒRun nc on server:”…””}”(hj¸"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M^hj§"h²hubj8 )”}”(hŒVnstatuser@nstat-b:~$ nc -lkv 0.0.0.0 9000 Listening on [0.0.0.0] (family 0, port 9000)”h]”hŒVnstatuser@nstat-b:~$ nc -lkv 0.0.0.0 9000 Listening on [0.0.0.0] (family 0, port 9000)”…””}”hjÆ"sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M`hj§"h²hubhÌ)”}”(hŒRun nc on client::”h]”hŒRun nc on client:”…””}”(hjÔ"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mchj§"h²hubj8 )”}”(hŒZnstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded!”h]”hŒZnstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded!”…””}”hjâ"sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mehj§"h²hubhÌ)”}”(hŒ:Input a string in the nc client ('hello' in our example)::”h]”hŒ=Input a string in the nc client (‘hello’ in our example):”…””}”(hjð"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhhj§"h²hubj8 )”}”(hŒ`nstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded! hello”h]”hŒ`nstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded! hello”…””}”hjþ"sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mjhj§"h²hubhÌ)”}”(hŒThe client side nstat output::”h]”hŒThe client side nstat output:”…””}”(hj #h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mnhj§"h²hubj8 )”}”(hXHnstatuser@nstat-a:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 TcpExtTCPPureAcks 1 0.0 TcpExtTCPOrigDataSent 1 0.0 IpExtInOctets 52 0.0 IpExtOutOctets 58 0.0 IpExtInNoECTPkts 1 0.0”h]”hXHnstatuser@nstat-a:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 TcpExtTCPPureAcks 1 0.0 TcpExtTCPOrigDataSent 1 0.0 IpExtInOctets 52 0.0 IpExtOutOctets 58 0.0 IpExtInNoECTPkts 1 0.0”…””}”hj#sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mphj§"h²hubhÌ)”}”(hŒThe server side nstat output::”h]”hŒThe server side nstat output:”…””}”(hj(#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M}hj§"h²hubj8 )”}”(hXÚnstatuser@nstat-b:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 IpExtInOctets 58 0.0 IpExtOutOctets 52 0.0 IpExtInNoECTPkts 1 0.0”h]”hXÚnstatuser@nstat-b:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 IpExtInOctets 58 0.0 IpExtOutOctets 52 0.0 IpExtInNoECTPkts 1 0.0”…””}”hj6#sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mhj§"h²hubhÌ)”}”(hŒAInput a string in nc client side again ('world' in our example)::”h]”hŒDInput a string in nc client side again (‘world’ in our example):”…””}”(hjD#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MŠhj§"h²hubj8 )”}”(hŒfnstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded! hello world”h]”hŒfnstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded! hello world”…””}”hjR#sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MŒhj§"h²hubhÌ)”}”(hŒClient side nstat output::”h]”hŒClient side nstat output:”…””}”(hj`#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M‘hj§"h²hubj8 )”}”(hXHnstatuser@nstat-a:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 TcpExtTCPHPAcks 1 0.0 TcpExtTCPOrigDataSent 1 0.0 IpExtInOctets 52 0.0 IpExtOutOctets 58 0.0 IpExtInNoECTPkts 1 0.0”h]”hXHnstatuser@nstat-a:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 TcpExtTCPHPAcks 1 0.0 TcpExtTCPOrigDataSent 1 0.0 IpExtInOctets 52 0.0 IpExtOutOctets 58 0.0 IpExtInNoECTPkts 1 0.0”…””}”hjn#sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M“hj§"h²hubhÌ)”}”(hŒServer side nstat output::”h]”hŒServer side nstat output:”…””}”(hj|#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¡hj§"h²hubj8 )”}”(hXnstatuser@nstat-b:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 TcpExtTCPHPHits 1 0.0 IpExtInOctets 58 0.0 IpExtOutOctets 52 0.0 IpExtInNoECTPkts 1 0.0”h]”hXnstatuser@nstat-b:~$ nstat #kernel IpInReceives 1 0.0 IpInDelivers 1 0.0 IpOutRequests 1 0.0 TcpInSegs 1 0.0 TcpOutSegs 1 0.0 TcpExtTCPHPHits 1 0.0 IpExtInOctets 58 0.0 IpExtOutOctets 52 0.0 IpExtInNoECTPkts 1 0.0”…””}”hjŠ#sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M£hj§"h²hubhÌ)”}”(hX¶Compare the first client-side nstat and the second client-side nstat, we could find one difference: the first one had a 'TcpExtTCPPureAcks', but the second one had a 'TcpExtTCPHPAcks'. The first server-side nstat and the second server-side nstat had a difference too: the second server-side nstat had a TcpExtTCPHPHits, but the first server-side nstat didn't have it. The network traffic patterns were exactly the same: the client sent a packet to the server, the server replied an ACK. But kernel handled them in different ways. When the TCP window scale option is not used, kernel will try to enable fast path immediately when the connection comes into the established state, but if the TCP window scale option is used, kernel will disable the fast path at first, and try to enable it after kernel receives packets. We could use the 'ss' command to verify whether the window scale option is used. e.g. run below command on either server or client::”h]”hXÃCompare the first client-side nstat and the second client-side nstat, we could find one difference: the first one had a ‘TcpExtTCPPureAcks’, but the second one had a ‘TcpExtTCPHPAcks’. The first server-side nstat and the second server-side nstat had a difference too: the second server-side nstat had a TcpExtTCPHPHits, but the first server-side nstat didn’t have it. The network traffic patterns were exactly the same: the client sent a packet to the server, the server replied an ACK. But kernel handled them in different ways. When the TCP window scale option is not used, kernel will try to enable fast path immediately when the connection comes into the established state, but if the TCP window scale option is used, kernel will disable the fast path at first, and try to enable it after kernel receives packets. We could use the ‘ss’ command to verify whether the window scale option is used. e.g. run below command on either server or client:”…””}”(hj˜#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¯hj§"h²hubj8 )”}”(hXnstatuser@nstat-a:~$ ss -o state established -i '( dport = :9000 or sport = :9000 ) Netid Recv-Q Send-Q Local Address:Port Peer Address:Port tcp 0 0 192.168.122.250:40654 192.168.122.251:9000 ts sack cubic wscale:7,7 rto:204 rtt:0.98/0.49 mss:1448 pmtu:1500 rcvmss:536 advmss:1448 cwnd:10 bytes_acked:1 segs_out:2 segs_in:1 send 118.2Mbps lastsnd:46572 lastrcv:46572 lastack:46572 pacing_rate 236.4Mbps rcv_space:29200 rcv_ssthresh:29200 minrtt:0.98”h]”hXnstatuser@nstat-a:~$ ss -o state established -i '( dport = :9000 or sport = :9000 ) Netid Recv-Q Send-Q Local Address:Port Peer Address:Port tcp 0 0 192.168.122.250:40654 192.168.122.251:9000 ts sack cubic wscale:7,7 rto:204 rtt:0.98/0.49 mss:1448 pmtu:1500 rcvmss:536 advmss:1448 cwnd:10 bytes_acked:1 segs_out:2 segs_in:1 send 118.2Mbps lastsnd:46572 lastrcv:46572 lastack:46572 pacing_rate 236.4Mbps rcv_space:29200 rcv_ssthresh:29200 minrtt:0.98”…””}”hj¦#sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M¿hj§"h²hubhÌ)”}”(hŒ‚The 'wscale:7,7' means both server and client set the window scale option to 7. Now we could explain the nstat output in our test:”h]”hŒ†The ‘wscale:7,7’ means both server and client set the window scale option to 7. Now we could explain the nstat output in our test:”…””}”(hj´#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÄhj§"h²hubhÌ)”}”(hŒÆIn the first nstat output of client side, the client sent a packet, server reply an ACK, when kernel handled this ACK, the fast path was not enabled, so the ACK was counted into 'TcpExtTCPPureAcks'.”h]”hŒÊIn the first nstat output of client side, the client sent a packet, server reply an ACK, when kernel handled this ACK, the fast path was not enabled, so the ACK was counted into ‘TcpExtTCPPureAcks’.”…””}”(hjÂ#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÇhj§"h²hubhÌ)”}”(hXIn the second nstat output of client side, the client sent a packet again, and received another ACK from the server, in this time, the fast path is enabled, and the ACK was qualified for fast path, so it was handled by the fast path, so this ACK was counted into TcpExtTCPHPAcks.”h]”hXIn the second nstat output of client side, the client sent a packet again, and received another ACK from the server, in this time, the fast path is enabled, and the ACK was qualified for fast path, so it was handled by the fast path, so this ACK was counted into TcpExtTCPHPAcks.”…””}”(hjÐ#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MËhj§"h²hubhÌ)”}”(hŒgIn the first nstat output of server side, fast path was not enabled, so there was no 'TcpExtTCPHPHits'.”h]”hŒkIn the first nstat output of server side, fast path was not enabled, so there was no ‘TcpExtTCPHPHits’.”…””}”(hjÞ#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÐhj§"h²hubhÌ)”}”(hŒ¬In the second nstat output of server side, the fast path was enabled, and the packet received from client qualified for fast path, so it was counted into 'TcpExtTCPHPHits'.”h]”hŒ°In the second nstat output of server side, the fast path was enabled, and the packet received from client qualified for fast path, so it was counted into ‘TcpExtTCPHPHits’.”…””}”(hjì#h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÓhj§"h²hubeh}”(h]”Œtcp-normal-traffic”ah ]”h"]”Œtcp normal traffic”ah$]”h&]”uh1hµhj!h²hh³hÊh´M]ubh¶)”}”(hhh]”(h»)”}”(hŒTcpExtTCPAbortOnClose”h]”hŒTcpExtTCPAbortOnClose”…””}”(hj$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj$h²hh³hÊh´MØubhÌ)”}”(hŒ0On the server side, we run below python script::”h]”hŒ/On the server side, we run below python script:”…””}”(hj$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÙhj$h²hubj8 )”}”(hŒ¿import socket import time port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(1) sock, addr = s.accept() while True: time.sleep(9999999)”h]”hŒ¿import socket import time port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(1) sock, addr = s.accept() while True: time.sleep(9999999)”…””}”hj!$sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MÛhj$h²hubhÌ)”}”(hŒVThis python script listen on 9000 port, but doesn't read anything from the connection.”h]”hŒXThis python script listen on 9000 port, but doesn’t read anything from the connection.”…””}”(hj/$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mçhj$h²hubhÌ)”}”(hŒ6On the client side, we send the string "hello" by nc::”h]”hŒ9On the client side, we send the string “hello†by nc:”…””}”(hj=$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mêhj$h²hubj8 )”}”(hŒ3nstatuser@nstat-a:~$ echo "hello" | nc nstat-b 9000”h]”hŒ3nstatuser@nstat-a:~$ echo "hello" | nc nstat-b 9000”…””}”hjK$sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mìhj$h²hubhÌ)”}”(hXThen, we come back to the server side, the server has received the "hello" packet, and the TCP layer has acked this packet, but the application didn't read it yet. We type Ctrl-C to terminate the server script. Then we could find TcpExtTCPAbortOnClose increased 1 on the server side::”h]”hX!Then, we come back to the server side, the server has received the “hello†packet, and the TCP layer has acked this packet, but the application didn’t read it yet. We type Ctrl-C to terminate the server script. Then we could find TcpExtTCPAbortOnClose increased 1 on the server side:”…””}”(hjY$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mîhj$h²hubj8 )”}”(hŒanstatuser@nstat-b:~$ nstat | grep -i abort TcpExtTCPAbortOnClose 1 0.0”h]”hŒanstatuser@nstat-b:~$ nstat | grep -i abort TcpExtTCPAbortOnClose 1 0.0”…””}”hjg$sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Móhj$h²hubhÌ)”}”(hŒ_If we run tcpdump on the server side, we could find the server sent a RST after we type Ctrl-C.”h]”hŒ_If we run tcpdump on the server side, we could find the server sent a RST after we type Ctrl-C.”…””}”(hju$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Möhj$h²hubeh}”(h]”Œtcpexttcpabortonclose”ah ]”h"]”Œtcpexttcpabortonclose”ah$]”h&]”uh1hµhj!h²hh³hÊh´MØubh¶)”}”(hhh]”(h»)”}”(hŒ2TcpExtTCPAbortOnMemory and TcpExtTCPAbortOnTimeout”h]”hŒ2TcpExtTCPAbortOnMemory and TcpExtTCPAbortOnTimeout”…””}”(hjŽ$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj‹$h²hh³hÊh´MúubhÌ)”}”(hŒ”Below is an example which let the orphan socket count be higher than net.ipv4.tcp_max_orphans. Change tcp_max_orphans to a smaller value on client::”h]”hŒ“Below is an example which let the orphan socket count be higher than net.ipv4.tcp_max_orphans. Change tcp_max_orphans to a smaller value on client:”…””}”(hjœ$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mûhj‹$h²hubj8 )”}”(hŒ;sudo bash -c "echo 10 > /proc/sys/net/ipv4/tcp_max_orphans"”h]”hŒ;sudo bash -c "echo 10 > /proc/sys/net/ipv4/tcp_max_orphans"”…””}”hjª$sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mÿhj‹$h²hubhÌ)”}”(hŒ.Client code (create 64 connection to server)::”h]”hŒ-Client code (create 64 connection to server):”…””}”(hj¸$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhj‹$h²hubj8 )”}”(hX}nstatuser@nstat-a:~$ cat client_orphan.py import socket import time server = 'nstat-b' # server address port = 9000 count = 64 connection_list = [] for i in range(64): s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((server, port)) connection_list.append(s) print("connection_count: %d" % len(connection_list)) while True: time.sleep(99999)”h]”hX}nstatuser@nstat-a:~$ cat client_orphan.py import socket import time server = 'nstat-b' # server address port = 9000 count = 64 connection_list = [] for i in range(64): s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((server, port)) connection_list.append(s) print("connection_count: %d" % len(connection_list)) while True: time.sleep(99999)”…””}”hjÆ$sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mhj‹$h²hubhÌ)”}”(hŒ0Server code (accept 64 connection from client)::”h]”hŒ/Server code (accept 64 connection from client):”…””}”(hjÔ$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhj‹$h²hubj8 )”}”(hX[nstatuser@nstat-b:~$ cat server_orphan.py import socket import time port = 9000 count = 64 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(count) connection_list = [] while True: sock, addr = s.accept() connection_list.append((sock, addr)) print("connection_count: %d" % len(connection_list))”h]”hX[nstatuser@nstat-b:~$ cat server_orphan.py import socket import time port = 9000 count = 64 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(count) connection_list = [] while True: sock, addr = s.accept() connection_list.append((sock, addr)) print("connection_count: %d" % len(connection_list))”…””}”hjâ$sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mhj‹$h²hubhÌ)”}”(hŒ,Run the python scripts on server and client.”h]”hŒ,Run the python scripts on server and client.”…””}”(hjð$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M)hj‹$h²hubhÌ)”}”(hŒ On server::”h]”hŒ On server:”…””}”(hjþ$h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M+hj‹$h²hubj8 )”}”(hŒpython3 server_orphan.py”h]”hŒpython3 server_orphan.py”…””}”hj %sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M-hj‹$h²hubhÌ)”}”(hŒ On client::”h]”hŒ On client:”…””}”(hj%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M/hj‹$h²hubj8 )”}”(hŒpython3 client_orphan.py”h]”hŒpython3 client_orphan.py”…””}”hj(%sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M1hj‹$h²hubhÌ)”}”(hŒRun iptables on server::”h]”hŒRun iptables on server:”…””}”(hj6%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M3hj‹$h²hubj8 )”}”(hŒEsudo iptables -A INPUT -i ens3 -p tcp --destination-port 9000 -j DROP”h]”hŒEsudo iptables -A INPUT -i ens3 -p tcp --destination-port 9000 -j DROP”…””}”hjD%sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M5hj‹$h²hubhÌ)”}”(hŒ-Type Ctrl-C on client, stop client_orphan.py.”h]”hŒ-Type Ctrl-C on client, stop client_orphan.py.”…””}”(hjR%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M7hj‹$h²hubhÌ)”}”(hŒ(Check TcpExtTCPAbortOnMemory on client::”h]”hŒ'Check TcpExtTCPAbortOnMemory on client:”…””}”(hj`%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M9hj‹$h²hubj8 )”}”(hŒanstatuser@nstat-a:~$ nstat | grep -i abort TcpExtTCPAbortOnMemory 54 0.0”h]”hŒanstatuser@nstat-a:~$ nstat | grep -i abort TcpExtTCPAbortOnMemory 54 0.0”…””}”hjn%sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M;hj‹$h²hubhÌ)”}”(hŒ'Check orphaned socket count on client::”h]”hŒ&Check orphaned socket count on client:”…””}”(hj|%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M>hj‹$h²hubj8 )”}”(hXanstatuser@nstat-a:~$ ss -s Total: 131 (kernel 0) TCP: 14 (estab 1, closed 0, orphaned 10, synrecv 0, timewait 0/0), ports 0 Transport Total IP IPv6 * 0 - - RAW 1 0 1 UDP 1 1 0 TCP 14 13 1 INET 16 14 2 FRAG 0 0 0”h]”hXanstatuser@nstat-a:~$ ss -s Total: 131 (kernel 0) TCP: 14 (estab 1, closed 0, orphaned 10, synrecv 0, timewait 0/0), ports 0 Transport Total IP IPv6 * 0 - - RAW 1 0 1 UDP 1 1 0 TCP 14 13 1 INET 16 14 2 FRAG 0 0 0”…””}”hjŠ%sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M@hj‹$h²hubhÌ)”}”(hX§The explanation of the test: after run server_orphan.py and client_orphan.py, we set up 64 connections between server and client. Run the iptables command, the server will drop all packets from the client, type Ctrl-C on client_orphan.py, the system of the client would try to close these connections, and before they are closed gracefully, these connections became orphan sockets. As the iptables of the server blocked packets from the client, the server won't receive fin from the client, so all connection on clients would be stuck on FIN_WAIT_1 stage, so they will keep as orphan sockets until timeout. We have echo 10 to /proc/sys/net/ipv4/tcp_max_orphans, so the client system would only keep 10 orphan sockets, for all other orphan sockets, the client system sent RST for them and delete them. We have 64 connections, so the 'ss -s' command shows the system has 10 orphan sockets, and the value of TcpExtTCPAbortOnMemory was 54.”h]”hX­The explanation of the test: after run server_orphan.py and client_orphan.py, we set up 64 connections between server and client. Run the iptables command, the server will drop all packets from the client, type Ctrl-C on client_orphan.py, the system of the client would try to close these connections, and before they are closed gracefully, these connections became orphan sockets. As the iptables of the server blocked packets from the client, the server won’t receive fin from the client, so all connection on clients would be stuck on FIN_WAIT_1 stage, so they will keep as orphan sockets until timeout. We have echo 10 to /proc/sys/net/ipv4/tcp_max_orphans, so the client system would only keep 10 orphan sockets, for all other orphan sockets, the client system sent RST for them and delete them. We have 64 connections, so the ‘ss -s’ command shows the system has 10 orphan sockets, and the value of TcpExtTCPAbortOnMemory was 54.”…””}”(hj˜%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MLhj‹$h²hubhÌ)”}”(hXšAn additional explanation about orphan socket count: You could find the exactly orphan socket count by the 'ss -s' command, but when kernel decide whither increases TcpExtTCPAbortOnMemory and sends RST, kernel doesn't always check the exactly orphan socket count. For increasing performance, kernel checks an approximate count firstly, if the approximate count is more than tcp_max_orphans, kernel checks the exact count again. So if the approximate count is less than tcp_max_orphans, but exactly count is more than tcp_max_orphans, you would find TcpExtTCPAbortOnMemory is not increased at all. If tcp_max_orphans is large enough, it won't occur, but if you decrease tcp_max_orphans to a small value like our test, you might find this issue. So in our test, the client set up 64 connections although the tcp_max_orphans is 10. If the client only set up 11 connections, we can't find the change of TcpExtTCPAbortOnMemory.”h]”hX¤An additional explanation about orphan socket count: You could find the exactly orphan socket count by the ‘ss -s’ command, but when kernel decide whither increases TcpExtTCPAbortOnMemory and sends RST, kernel doesn’t always check the exactly orphan socket count. For increasing performance, kernel checks an approximate count firstly, if the approximate count is more than tcp_max_orphans, kernel checks the exact count again. So if the approximate count is less than tcp_max_orphans, but exactly count is more than tcp_max_orphans, you would find TcpExtTCPAbortOnMemory is not increased at all. If tcp_max_orphans is large enough, it won’t occur, but if you decrease tcp_max_orphans to a small value like our test, you might find this issue. So in our test, the client set up 64 connections although the tcp_max_orphans is 10. If the client only set up 11 connections, we can’t find the change of TcpExtTCPAbortOnMemory.”…””}”(hj¦%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M[hj‹$h²hubhÌ)”}”(hX.Continue the previous test, we wait for several minutes. Because of the iptables on the server blocked the traffic, the server wouldn't receive fin, and all the client's orphan sockets would timeout on the FIN_WAIT_1 state finally. So we wait for a few minutes, we could find 10 timeout on the client::”h]”hX1Continue the previous test, we wait for several minutes. Because of the iptables on the server blocked the traffic, the server wouldn’t receive fin, and all the client’s orphan sockets would timeout on the FIN_WAIT_1 state finally. So we wait for a few minutes, we could find 10 timeout on the client:”…””}”(hj´%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mjhj‹$h²hubj8 )”}”(hŒanstatuser@nstat-a:~$ nstat | grep -i abort TcpExtTCPAbortOnTimeout 10 0.0”h]”hŒanstatuser@nstat-a:~$ nstat | grep -i abort TcpExtTCPAbortOnTimeout 10 0.0”…””}”hjÂ%sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mphj‹$h²hubeh}”(h]”Œ2tcpexttcpabortonmemory-and-tcpexttcpabortontimeout”ah ]”h"]”Œ2tcpexttcpabortonmemory and tcpexttcpabortontimeout”ah$]”h&]”uh1hµhj!h²hh³hÊh´Múubh¶)”}”(hhh]”(h»)”}”(hŒTcpExtTCPAbortOnLinger”h]”hŒTcpExtTCPAbortOnLinger”…””}”(hjÛ%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjØ%h²hh³hÊh´MtubhÌ)”}”(hŒThe server side code::”h]”hŒThe server side code:”…””}”(hjé%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MuhjØ%h²hubj8 )”}”(hŒénstatuser@nstat-b:~$ cat server_linger.py import socket import time port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(1) sock, addr = s.accept() while True: time.sleep(9999999)”h]”hŒénstatuser@nstat-b:~$ cat server_linger.py import socket import time port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(1) sock, addr = s.accept() while True: time.sleep(9999999)”…””}”hj÷%sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MwhjØ%h²hubhÌ)”}”(hŒThe client side code::”h]”hŒThe client side code:”…””}”(hj&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M„hjØ%h²hubj8 )”}”(hXdnstatuser@nstat-a:~$ cat client_linger.py import socket import struct server = 'nstat-b' # server address port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 10)) s.setsockopt(socket.SOL_TCP, socket.TCP_LINGER2, struct.pack('i', -1)) s.connect((server, port)) s.close()”h]”hXdnstatuser@nstat-a:~$ cat client_linger.py import socket import struct server = 'nstat-b' # server address port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 10)) s.setsockopt(socket.SOL_TCP, socket.TCP_LINGER2, struct.pack('i', -1)) s.connect((server, port)) s.close()”…””}”hj&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M†hjØ%h²hubhÌ)”}”(hŒ Run server_linger.py on server::”h]”hŒRun server_linger.py on server:”…””}”(hj!&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M“hjØ%h²hubj8 )”}”(hŒ-nstatuser@nstat-b:~$ python3 server_linger.py”h]”hŒ-nstatuser@nstat-b:~$ python3 server_linger.py”…””}”hj/&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M•hjØ%h²hubhÌ)”}”(hŒ Run client_linger.py on client::”h]”hŒRun client_linger.py on client:”…””}”(hj=&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M—hjØ%h²hubj8 )”}”(hŒ-nstatuser@nstat-a:~$ python3 client_linger.py”h]”hŒ-nstatuser@nstat-a:~$ python3 client_linger.py”…””}”hjK&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M™hjØ%h²hubhÌ)”}”(hŒ7After run client_linger.py, check the output of nstat::”h]”hŒ6After run client_linger.py, check the output of nstat:”…””}”(hjY&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M›hjØ%h²hubj8 )”}”(hŒanstatuser@nstat-a:~$ nstat | grep -i abort TcpExtTCPAbortOnLinger 1 0.0”h]”hŒanstatuser@nstat-a:~$ nstat | grep -i abort TcpExtTCPAbortOnLinger 1 0.0”…””}”hjg&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MhjØ%h²hubeh}”(h]”Œtcpexttcpabortonlinger”ah ]”h"]”Œtcpexttcpabortonlinger”ah$]”h&]”uh1hµhj!h²hh³hÊh´Mtubh¶)”}”(hhh]”(h»)”}”(hŒTcpExtTCPRcvCoalesce”h]”hŒTcpExtTCPRcvCoalesce”…””}”(hj€&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj}&h²hh³hÊh´M¡ubhÌ)”}”(hŒZOn the server, we run a program which listen on TCP port 9000, but doesn't read any data::”h]”hŒ[On the server, we run a program which listen on TCP port 9000, but doesn’t read any data:”…””}”(hjŽ&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¢hj}&h²hubj8 )”}”(hŒ½import socket import time port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(1) sock, addr = s.accept() while True: time.sleep(9999999)”h]”hŒ½import socket import time port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('0.0.0.0', port)) s.listen(1) sock, addr = s.accept() while True: time.sleep(9999999)”…””}”hjœ&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M¥hj}&h²hubhÌ)”}”(hŒ4Save the above code as server_coalesce.py, and run::”h]”hŒ3Save the above code as server_coalesce.py, and run:”…””}”(hjª&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¯hj}&h²hubj8 )”}”(hŒpython3 server_coalesce.py”h]”hŒpython3 server_coalesce.py”…””}”hj¸&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M±hj}&h²hubhÌ)”}”(hŒ6On the client, save below code as client_coalesce.py::”h]”hŒ5On the client, save below code as client_coalesce.py:”…””}”(hjÆ&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M³hj}&h²hubj8 )”}”(hŒ|import socket server = 'nstat-b' port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((server, port))”h]”hŒ|import socket server = 'nstat-b' port = 9000 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((server, port))”…””}”hjÔ&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mµhj}&h²hubhÌ)”}”(hŒRun::”h]”hŒRun:”…””}”(hjâ&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M»hj}&h²hubj8 )”}”(hŒ2nstatuser@nstat-a:~$ python3 -i client_coalesce.py”h]”hŒ2nstatuser@nstat-a:~$ python3 -i client_coalesce.py”…””}”hjð&sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M½hj}&h²hubhÌ)”}”(hŒ>We use '-i' to come into the interactive mode, then a packet::”h]”hŒAWe use ‘-i’ to come into the interactive mode, then a packet:”…””}”(hjþ&h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M¿hj}&h²hubj8 )”}”(hŒ>>> s.send(b'foo') 3”h]”hŒ>>> s.send(b'foo') 3”…””}”hj 'sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MÁhj}&h²hubhÌ)”}”(hŒSend a packet again::”h]”hŒSend a packet again:”…””}”(hj'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÄhj}&h²hubj8 )”}”(hŒ>>> s.send(b'bar') 3”h]”hŒ>>> s.send(b'bar') 3”…””}”hj('sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MÆhj}&h²hubhÌ)”}”(hŒOn the server, run nstat::”h]”hŒOn the server, run nstat:”…””}”(hj6'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÉhj}&h²hubj8 )”}”(hXubuntu@nstat-b:~$ nstat #kernel IpInReceives 2 0.0 IpInDelivers 2 0.0 IpOutRequests 2 0.0 TcpInSegs 2 0.0 TcpOutSegs 2 0.0 TcpExtTCPRcvCoalesce 1 0.0 IpExtInOctets 110 0.0 IpExtOutOctets 104 0.0 IpExtInNoECTPkts 2 0.0”h]”hXubuntu@nstat-b:~$ nstat #kernel IpInReceives 2 0.0 IpInDelivers 2 0.0 IpOutRequests 2 0.0 TcpInSegs 2 0.0 TcpOutSegs 2 0.0 TcpExtTCPRcvCoalesce 1 0.0 IpExtInOctets 110 0.0 IpExtOutOctets 104 0.0 IpExtInNoECTPkts 2 0.0”…””}”hjD'sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MËhj}&h²hubhÌ)”}”(hŒ÷The client sent two packets, server didn't read any data. When the second packet arrived at server, the first packet was still in the receiving queue. So the TCP layer merged the two packets, and we could find the TcpExtTCPRcvCoalesce increased 1.”h]”hŒùThe client sent two packets, server didn’t read any data. When the second packet arrived at server, the first packet was still in the receiving queue. So the TCP layer merged the two packets, and we could find the TcpExtTCPRcvCoalesce increased 1.”…””}”(hjR'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M×hj}&h²hubeh}”(h]”Œtcpexttcprcvcoalesce”ah ]”h"]”Œtcpexttcprcvcoalesce”ah$]”h&]”uh1hµhj!h²hh³hÊh´M¡ubh¶)”}”(hhh]”(h»)”}”(hŒ+TcpExtListenOverflows and TcpExtListenDrops”h]”hŒ+TcpExtListenOverflows and TcpExtListenDrops”…””}”(hjk'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjh'h²hh³hÊh´MÝubhÌ)”}”(hŒ4On server, run the nc command, listen on port 9000::”h]”hŒ3On server, run the nc command, listen on port 9000:”…””}”(hjy'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MÞhjh'h²hubj8 )”}”(hŒVnstatuser@nstat-b:~$ nc -lkv 0.0.0.0 9000 Listening on [0.0.0.0] (family 0, port 9000)”h]”hŒVnstatuser@nstat-b:~$ nc -lkv 0.0.0.0 9000 Listening on [0.0.0.0] (family 0, port 9000)”…””}”hj‡'sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Màhjh'h²hubhÌ)”}”(hŒ5On client, run 3 nc commands in different terminals::”h]”hŒ4On client, run 3 nc commands in different terminals:”…””}”(hj•'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mãhjh'h²hubj8 )”}”(hŒZnstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded!”h]”hŒZnstatuser@nstat-a:~$ nc -v nstat-b 9000 Connection to nstat-b 9000 port [tcp/*] succeeded!”…””}”hj£'sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Måhjh'h²hubhÌ)”}”(hX The nc command only accepts 1 connection, and the accept queue length is 1. On current linux implementation, set queue length to n means the actual queue length is n+1. Now we create 3 connections, 1 is accepted by nc, 2 in accepted queue, so the accept queue is full.”h]”hX The nc command only accepts 1 connection, and the accept queue length is 1. On current linux implementation, set queue length to n means the actual queue length is n+1. Now we create 3 connections, 1 is accepted by nc, 2 in accepted queue, so the accept queue is full.”…””}”(hj±'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mèhjh'h²hubhÌ)”}”(hŒEBefore running the 4th nc, we clean the nstat history on the server::”h]”hŒDBefore running the 4th nc, we clean the nstat history on the server:”…””}”(hj¿'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Míhjh'h²hubj8 )”}”(hŒnstatuser@nstat-b:~$ nstat -n”h]”hŒnstatuser@nstat-b:~$ nstat -n”…””}”hjÍ'sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mïhjh'h²hubhÌ)”}”(hŒRun the 4th nc on the client::”h]”hŒRun the 4th nc on the client:”…””}”(hjÛ'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mñhjh'h²hubj8 )”}”(hŒ'nstatuser@nstat-a:~$ nc -v nstat-b 9000”h]”hŒ'nstatuser@nstat-a:~$ nc -v nstat-b 9000”…””}”hjé'sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Móhjh'h²hubhÌ)”}”(hX®If the nc server is running on kernel 4.10 or higher version, you won't see the "Connection to ... succeeded!" string, because kernel will drop the SYN if the accept queue is full. If the nc client is running on an old kernel, you would see that the connection is succeeded, because kernel would complete the 3 way handshake and keep the socket on half open queue. I did the test on kernel 4.15. Below is the nstat on the server::”h]”hX³If the nc server is running on kernel 4.10 or higher version, you won’t see the “Connection to ... succeeded!†string, because kernel will drop the SYN if the accept queue is full. If the nc client is running on an old kernel, you would see that the connection is succeeded, because kernel would complete the 3 way handshake and keep the socket on half open queue. I did the test on kernel 4.15. Below is the nstat on the server:”…””}”(hj÷'h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mõhjh'h²hubj8 )”}”(hX£nstatuser@nstat-b:~$ nstat #kernel IpInReceives 4 0.0 IpInDelivers 4 0.0 TcpInSegs 4 0.0 TcpExtListenOverflows 4 0.0 TcpExtListenDrops 4 0.0 IpExtInOctets 240 0.0 IpExtInNoECTPkts 4 0.0”h]”hX£nstatuser@nstat-b:~$ nstat #kernel IpInReceives 4 0.0 IpInDelivers 4 0.0 TcpInSegs 4 0.0 TcpExtListenOverflows 4 0.0 TcpExtListenDrops 4 0.0 IpExtInOctets 240 0.0 IpExtInNoECTPkts 4 0.0”…””}”hj(sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mýhjh'h²hubhÌ)”}”(hŒÿBoth TcpExtListenOverflows and TcpExtListenDrops were 4. If the time between the 4th nc and the nstat was longer, the value of TcpExtListenOverflows and TcpExtListenDrops would be larger, because the SYN of the 4th nc was dropped, the client was retrying.”h]”hŒÿBoth TcpExtListenOverflows and TcpExtListenDrops were 4. If the time between the 4th nc and the nstat was longer, the value of TcpExtListenOverflows and TcpExtListenDrops would be larger, because the SYN of the 4th nc was dropped, the client was retrying.”…””}”(hj(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhjh'h²hubeh}”(h]”Œ+tcpextlistenoverflows-and-tcpextlistendrops”ah ]”h"]”Œ+tcpextlistenoverflows and tcpextlistendrops”ah$]”h&]”uh1hµhj!h²hh³hÊh´MÝubh¶)”}”(hhh]”(h»)”}”(hŒ1IpInAddrErrors, IpExtInNoRoutes and IpOutNoRoutes”h]”hŒ1IpInAddrErrors, IpExtInNoRoutes and IpOutNoRoutes”…””}”(hj,(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj)(h²hh³hÊh´M ubhÌ)”}”(hŒxserver A IP address: 192.168.122.250 server B IP address: 192.168.122.251 Prepare on server A, add a route to server B::”h]”hŒwserver A IP address: 192.168.122.250 server B IP address: 192.168.122.251 Prepare on server A, add a route to server B:”…””}”(hj:(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhj)(h²hubj8 )”}”(hŒ2$ sudo ip route add 8.8.8.8/32 via 192.168.122.251”h]”hŒ2$ sudo ip route add 8.8.8.8/32 via 192.168.122.251”…””}”hjH(sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mhj)(h²hubhÌ)”}”(hŒ@Prepare on server B, disable send_redirects for all interfaces::”h]”hŒ?Prepare on server B, disable send_redirects for all interfaces:”…””}”(hjV(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhj)(h²hubj8 )”}”(hŒÓ$ sudo sysctl -w net.ipv4.conf.all.send_redirects=0 $ sudo sysctl -w net.ipv4.conf.ens3.send_redirects=0 $ sudo sysctl -w net.ipv4.conf.lo.send_redirects=0 $ sudo sysctl -w net.ipv4.conf.default.send_redirects=0”h]”hŒÓ$ sudo sysctl -w net.ipv4.conf.all.send_redirects=0 $ sudo sysctl -w net.ipv4.conf.ens3.send_redirects=0 $ sudo sysctl -w net.ipv4.conf.lo.send_redirects=0 $ sudo sysctl -w net.ipv4.conf.default.send_redirects=0”…””}”hjd(sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´Mhj)(h²hubhÌ)”}”(hŒÝWe want to let sever A send a packet to 8.8.8.8, and route the packet to server B. When server B receives such packet, it might send a ICMP Redirect message to server A, set send_redirects to 0 will disable this behavior.”•³vh]”hŒÝWe want to let sever A send a packet to 8.8.8.8, and route the packet to server B. When server B receives such packet, it might send a ICMP Redirect message to server A, set send_redirects to 0 will disable this behavior.”…””}”(hjr(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Mhj)(h²hubhÌ)”}”(hŒEFirst, generate InAddrErrors. On server B, we disable IP forwarding::”h]”hŒDFirst, generate InAddrErrors. On server B, we disable IP forwarding:”…””}”(hj€(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M hj)(h²hubj8 )”}”(hŒ/$ sudo sysctl -w net.ipv4.conf.all.forwarding=0”h]”hŒ/$ sudo sysctl -w net.ipv4.conf.all.forwarding=0”…””}”hjŽ(sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M"hj)(h²hubhÌ)”}”(hŒ)On server A, we send packets to 8.8.8.8::”h]”hŒ(On server A, we send packets to 8.8.8.8:”…””}”(hjœ(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M$hj)(h²hubj8 )”}”(hŒ$ nc -v 8.8.8.8 53”h]”hŒ$ nc -v 8.8.8.8 53”…””}”hjª(sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M&hj)(h²hubhÌ)”}”(hŒ+On server B, we check the output of nstat::”h]”hŒ*On server B, we check the output of nstat:”…””}”(hj¸(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M(hj)(h²hubj8 )”}”(hŒë$ nstat #kernel IpInReceives 3 0.0 IpInAddrErrors 3 0.0 IpExtInOctets 180 0.0 IpExtInNoECTPkts 3 0.0”h]”hŒë$ nstat #kernel IpInReceives 3 0.0 IpInAddrErrors 3 0.0 IpExtInOctets 180 0.0 IpExtInNoECTPkts 3 0.0”…””}”hjÆ(sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M*hj)(h²hubhÌ)”}”(hX2As we have let server A route 8.8.8.8 to server B, and we disabled IP forwarding on server B, Server A sent packets to server B, then server B dropped packets and increased IpInAddrErrors. As the nc command would re-send the SYN packet if it didn't receive a SYN+ACK, we could find multiple IpInAddrErrors.”h]”hX4As we have let server A route 8.8.8.8 to server B, and we disabled IP forwarding on server B, Server A sent packets to server B, then server B dropped packets and increased IpInAddrErrors. As the nc command would re-send the SYN packet if it didn’t receive a SYN+ACK, we could find multiple IpInAddrErrors.”…””}”(hjÔ(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M1hj)(h²hubhÌ)”}”(hŒHSecond, generate IpExtInNoRoutes. On server B, we enable IP forwarding::”h]”hŒGSecond, generate IpExtInNoRoutes. On server B, we enable IP forwarding:”…””}”(hjâ(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M7hj)(h²hubj8 )”}”(hŒ/$ sudo sysctl -w net.ipv4.conf.all.forwarding=1”h]”hŒ/$ sudo sysctl -w net.ipv4.conf.all.forwarding=1”…””}”hjð(sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M:hj)(h²hubhÌ)”}”(hŒ@Check the route table of server B and remove the default route::”h]”hŒ?Check the route table of server B and remove the default route:”…””}”(hjþ(h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´M<hj)(h²hubj8 )”}”(hŒÌ$ ip route show default via 192.168.122.1 dev ens3 proto static 192.168.122.0/24 dev ens3 proto kernel scope link src 192.168.122.251 $ sudo ip route delete default via 192.168.122.1 dev ens3 proto static”h]”hŒÌ$ ip route show default via 192.168.122.1 dev ens3 proto static 192.168.122.0/24 dev ens3 proto kernel scope link src 192.168.122.251 $ sudo ip route delete default via 192.168.122.1 dev ens3 proto static”…””}”hj )sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´M>hj)(h²hubhÌ)”}”(hŒ'On server A, we contact 8.8.8.8 again::”h]”hŒ&On server A, we contact 8.8.8.8 again:”…””}”(hj)h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MChj)(h²hubj8 )”}”(hŒV$ nc -v 8.8.8.8 53 nc: connect to 8.8.8.8 port 53 (tcp) failed: Network is unreachable”h]”hŒV$ nc -v 8.8.8.8 53 nc: connect to 8.8.8.8 port 53 (tcp) failed: Network is unreachable”…””}”hj()sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MEhj)(h²hubhÌ)”}”(hŒOn server B, run nstat::”h]”hŒOn server B, run nstat:”…””}”(hj6)h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MHhj)(h²hubj8 )”}”(hXþ$ nstat #kernel IpInReceives 1 0.0 IpOutRequests 1 0.0 IcmpOutMsgs 1 0.0 IcmpOutDestUnreachs 1 0.0 IcmpMsgOutType3 1 0.0 IpExtInNoRoutes 1 0.0 IpExtInOctets 60 0.0 IpExtOutOctets 88 0.0 IpExtInNoECTPkts 1 0.0”h]”hXþ$ nstat #kernel IpInReceives 1 0.0 IpOutRequests 1 0.0 IcmpOutMsgs 1 0.0 IcmpOutDestUnreachs 1 0.0 IcmpMsgOutType3 1 0.0 IpExtInNoRoutes 1 0.0 IpExtInOctets 60 0.0 IpExtOutOctets 88 0.0 IpExtInNoECTPkts 1 0.0”…””}”hjD)sbah}”(h]”h ]”h"]”h$]”h&]”jG jH uh1j7 h³hÊh´MJhj)(h²hubhÌ)”}”(hXAWe enabled IP forwarding on server B, when server B received a packet which destination IP address is 8.8.8.8, server B will try to forward this packet. We have deleted the default route, there was no route for 8.8.8.8, so server B increase IpExtInNoRoutes and sent the "ICMP Destination Unreachable" message to server A.”h]”hXEWe enabled IP forwarding on server B, when server B received a packet which destination IP address is 8.8.8.8, server B will try to forward this packet. We have deleted the default route, there was no route for 8.8.8.8, so server B increase IpExtInNoRoutes and sent the “ICMP Destination Unreachable†message to server A.”…””}”(hjR)h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´MVhj)(h²hubhÌ)”}”(hŒ=Third, generate IpOutNoRoutes. Run ping command on server B::”h]”hŒ jP jJ j\ jV jº j´ jX jR jd j^ j jx jj" jy js jÓ jÍ j- j' j‡ j jèjjcj]j³j­jjýjàjÚj‰jîj]jjêjäj…jjUjOjajcjj—j½jgjÓjÍjßjÙjÁjÃjýj÷jFjÇjújLj jjgj¦jõjïjjûj@jmj§j¡j jj,j&j8j2jæjFj·jìj"jj]j½j÷jñjÍjcjj—j¢jÓj jj' j¨j !j- j,j!jî!j!!j¡"jô!jü#j§"j…$j$jÒ%j‹$jw&jØ%jb'j}&j#(jh'j¨)j)(j¡*j®)j~+j§*j…,j„+uŒ footnote_refs”}”Œ citation_refs”}”Œ autofootnotes”]”Œautofootnote_refs”]”Œsymbol_footnotes”]”Œsymbol_footnote_refs”]”Œ footnotes”]”Œ citations”]”Œautofootnote_start”KŒsymbol_footnote_start”KŒ id_counter”Œ collections”ŒCounter”“”}”…”R”Œparse_messages”]”hŒsystem_message”“”)”}”(hhh]”hÌ)”}”(hŒ:Enumerated list start value not ordinal-1: "2" (ordinal 2)”h]”hŒ>Enumerated list start value not ordinal-1: “2†(ordinal 2)”…””}”(hj¡-h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËhjž-ubah}”(h]”h ]”h"]”h$]”h&]”Œlevel”KŒtype”ŒINFO”Œsource”hÊŒline”Kuh1jœ-hjh²hh³hÊh´MúubaŒtransform_messages”]”Œ transformer”NŒ include_log”]”Œ decoration”Nh²hub.