Oversized packets / FCS errors
This issue has something to do with jumbo frames I suppose. Of course in a SAN environment you want to use jumbo frames. The configuration on our switchesis done via the system setting 'system jumbo mtu 9216 '. So jumbo frames are enabled for all ports. But because we have a converged network this might not be desired?
Port-channel 24 is the connection between the DataCenter network and the Campus network. On the campus network switch, jumbo frames are not enabled because clients cannot benefit jumbo frames. This might be an issue. The Datacenter network is using jumbo frames when sending data to the campus network, but our Campus switch will need to split the packets. This may cause the high number of oversized packets being dropped.
On the N4000 side:
DataCenterCore#show interfaces counters port-channel 24
Alignment Errors: ............................. 0
FCS Errors: ................................... 0
Single Collision Frames: ...................... 0
Multiple Collision Frames: .................... 0
Late Collisions: .............................. 0
Excessive Collisions: ......................... 0
Received packets dropped > MTU: ............... 249656583
Transmitted packets dropped > MTU: ............ 720773428
Internal MAC Rx Errors: ....................... 0
Received Pause Frames: ........................ 0
Transmitted Pause Frames: ..................... 0
6224 side:
LANStack2#show interfaces counters port-channel 24
Alignment Errors: ............................. 0
FCS Errors: ................................... 4326
Single Collision Frames: ...................... 0
Multiple Collision Frames: .................... 0
Late Collisions: .............................. 0
Excessive Collisions: ......................... 0
Oversize Packets: ............................. 720759490
Internal MAC Rx Errors: ....................... 4326
Received Pause Frames: ........................ 0
Transmitted Pause Frames: ..................... 0
FCS errors are most likely layer 1 issues, according to my ccnp troubleshooting book. The ports are connected by fiber cabled with are not easy to change.... So my first thoughts were heading to the oversized packets.
Flow control is enabled which should segments packets in a acceptable size:
N4000
Port Description Vlan Duplex Speed Neg Link Flow Ctrl
State Status
--------- ------------------------- ----- ------ ------- ---- ------ ---------
Te1/0/24 gnrl Full 1000 Auto Up Active
Te2/0/24 gnrl Full 1000 Auto Up Active
6224
Port Type Duplex Speed Neg Link Flow Control
State Status
----- ------------------------------ ------ ------- ---- --------- ---------
1/g24 Gigabit - Level Full 1000 Auto Up Active
2/g24 Gigabit - Level Full 1000 Auto Up Active
Port configurations:
N4000:
DataCenterCore#show running-config interface te1/0/24
channel-group 24 mode active
DataCenterCore#show running-config interface te2/0/24
channel-group 24 mode active
DataCenterCore#show running-config interface port-channel 24
switchport mode general
switchport general allowed vlan add 2-7 tagged
6224:
LANStack2#show running-config interface ethernet 1/g24
channel-group 24 mode auto
LANStack2#show running-config interface ethernet 2/g24
channel-group 24 mode auto
LANStack2#show running-config interface port-channel 24
switchport mode general
switchport general allowed vlan add 2-7 tagged
Any comments on this configuration or behavior?
Full logs (after about 30 minutes after clearing the counters):
N4000
DataCenterCore#show interfaces counters port-channel 24
Ch InTotalPkts InUcastPkts InMcastPkts InBcastPkts
--------- ---------------- ---------------- ---------------- ----------------
Po24 6467205 6457388 2557 7260
Ch OutTotalPkts OutUcastPkts OutMcastPkts OutBcastPkts
--------- ---------------- ---------------- ---------------- ----------------
Po24 7721819 7677804 33410 10605
Alignment Errors: ............................. 0
FCS Errors: ................................... 0
Single Collision Frames: ...................... 0
Multiple Collision Frames: .................... 0
Late Collisions: .............................. 0
Excessive Collisions: ......................... 0
Received packets dropped > MTU: ............... 714215
Transmitted packets dropped > MTU: ............ 2949820
Internal MAC Rx Errors: ....................... 0
Received Pause Frames: ........................ 0
Transmitted Pause Frames: ..................... 0
DataCenterCore#show statistics port-channel 24
Total Packets Received (Octets)................ 2551407247
Packets Received 64 Octets..................... 328102
Packets Received 65-127 Octets................. 3196193
Packets Received 128-255 Octets................ 1134654
Packets Received 256-511 Octets................ 454190
Packets Received 512-1023 Octets............... 196276
Packets Received 1024-1518 Octets.............. 472093
Packets Received > 1518 Octets................. 718238
Packets RX and TX 64 Octets.................... 1117076
Packets RX and TX 65-127 Octets................ 4437329
Packets RX and TX 128-255 Octets............... 2002462
Packets RX and TX 256-511 Octets............... 1052252
Packets RX and TX 512-1023 Octets.............. 539752
Packets RX and TX 1024-1518 Octets............. 1431931
Packets RX and TX 1519-2047 Octets............. 0
Packets RX and TX 2048-4095 Octets............. 0
Packets RX and TX 4096-9216 Octets............. 20
Total Packets Received Without Errors.......... 6499659
Unicast Packets Received....................... 6489679
Multicast Packets Received..................... 2571
Broadcast Packets Received..................... 7409
Receive Packets Discarded...................... 0
Total Packets Received with MAC Errors......... 0
Jabbers Received............................... 0
Fragments/Undersize Received................... 0
Alignment Errors............................... 0
FCS Errors..................................... 0
Overruns....................................... 0
Total Received Packets Not Forwarded........... 0
802.3x Pause Frames Received................... 0
Unacceptable Frame Type........................ 0
Total Packets Transmitted (Octets)............. 6640832478
Packets Transmitted 64 Octets.................. 788974
Packets Transmitted 65-127 Octets.............. 1241136
Packets Transmitted 128-255 Octets............. 867808
Packets Transmitted 256-511 Octets............. 598062
Packets Transmitted 512-1023 Octets............ 343476
Packets Transmitted 1024-1518 Octets........... 959838
Packets Transmitted > 1518 Octets.............. 2951642
Max Frame Size................................. 9216
Total Packets Transmitted Successfully......... 7750956
Unicast Packets Transmitted.................... 7706688
Multicast Packets Transmitted.................. 33631
Broadcast Packets Transmitted.................. 10637
Transmit Packets Discarded..................... 0
Total Transmit Errors.......................... 0
FCS Errors..................................... 0
Underrun Errors................................ 0
Total Transmit Packets Discarded............... 0
Single Collision Frames........................ 0
Multiple Collision Frames...................... 0
Excessive Collision Frames..................... 0
802.3x Pause Frames Transmitted................ 0
GVRP PDUs received............................. 0
GVRP PDUs Transmitted.......................... 0
GVRP Failed Registrations...................... 0
GMRP PDUs Received............................. 0
GMRP PDUs Transmitted.......................... 0
GMRP Failed Registrations...................... 0
BPDU: sent 0, received 799
Time since counters last cleared............... 0 day 0 hr 26 min 37 sec
6224
LANStack2#show interfaces counters port-channel 24
Ch InOctets InUcastPkts InMcastPkts InBcastPkts
---------------- ---------- ----------- ----------- -----------
ch24 6741672293 7921528 35012 10809
Ch OutOctets OutUcastPkts OutMcastPkts OutBcastPkts
---------------- ---------- ------------ ------------ ------------
ch24 2644530397 6725600 2779 7747
Alignment Errors: ............................. 0
FCS Errors: ................................... 20
Single Collision Frames: ...................... 0
Multiple Collision Frames: .................... 0
Late Collisions: .............................. 0
Excessive Collisions: ......................... 0
Oversize Packets: ............................. 2972957
Internal MAC Rx Errors: ....................... 20
Received Pause Frames: ........................ 0
Transmitted Pause Frames: ..................... 0
LANStack2#show statistics port-channel 24
Total Packets Received (Octets)................ 6750962384
Packets Received > 1518 Octets................. 2974969
Packets RX and TX 64 Octets.................... 1156974
Packets RX and TX 65-127 Octets................ 4612545
Packets RX and TX 128-255 Octets............... 2114583
Packets RX and TX 256-511 Octets............... 1110681
Packets RX and TX 512-1023 Octets.............. 564669
Packets RX and TX 1024-1518 Octets............. 1471257
Packets RX and TX 1519-2047 Octets............. 3725801
Packets RX and TX 2048-4095 Octets............. 0
Packets RX and TX 4096-9216 Octets............. 0
Total Packets Received Without Errors.......... 7991675
Unicast Packets Received....................... 7945620
Multicast Packets Received..................... 35218
Broadcast Packets Received..................... 10837
Receive Packets Discarded...................... 0
Total Packets Received with MAC Errors......... 20
Jabbers Received............................... 0
Fragments/Undersize Received................... 0
Alignment Errors............................... 0
--More-- or (q)uit
FCS Errors..................................... 20
Overruns....................................... 0
802.3x Pause Frames Received................... 0
Unacceptable Frame Type........................ 0
Total Packets Transmitted (Octets)............. 2656984071
Packets Transmitted > 1518 Octets ............. 750832
Max Frame Size................................. 1518
Total Packets Transmitted Successfully......... 6764815
Unicast Packets Transmitted.................... 6755191
Multicast Packets Transmitted.................. 2806
Broadcast Packets Transmitted.................. 7802
Transmit Packets Discarded..................... 0
Total Transmit Errors.......................... 0
FCS Errors..................................... 0
Underrun Errors................................ 0
Total Transmit Packets Discarded............... 0
Single Collision Frames........................ 0
Multiple Collision Frames...................... 0
Excessive Collision Frames..................... 0
--More-- or (q)uit
802.3x Pause Frames Transmitted................ 0
GVRP PDUs received............................. 0
GVRP PDUs Transmitted.......................... 0
GVRP Failed Registrations...................... 0
BPDU: sent 837, received 0
Time Since Counters Last Cleared............... 0 day 0 hr 27 min 55 sec
Jumbo frames on ESX connected hosts
In relation to this issue I have another question. How will the switch handle non-jumbo frame received on the ethernet ports?
Like I said we have a converged network, meaning that LAN and SAN traffic is going over the same physical cable. iSCSI traffic is using jumbo frames as configured on the vmware site, but LAN traffic is not.