Dell R410 网卡在Centos5.5 上丢包

现象是每几十次访问网页就会一次打不开,数据库,临控等才时常取不到数据. 好像r410的网卡和centos5.5不怎么兼容

uname -a

Linux c1gserver 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

dmesg |grep bnx

Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.9.3 (March 17, 2009) bnx2: eth0: using MSIX bnx2: eth0 NIC Copper Link is Up, 100 Mbps full duplex

ifconfig 有大量的丢包;dropped:6620400

eth0 Link encap:Ethernet HWaddr 00:26:B9:3B:XX:C3 inet addr:61.255.xx.xx Bcast:61.255.xx.xx Mask:255.255.255.192 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7330597243 errors:1 dropped:6620400 overruns:0 frame:1 TX packets:8116489614 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3435468718053 (3.1 TiB) TX bytes:3656517601592 (3.3 TiB) Interrupt:82 Memory:da000000-da012800

一.先尝试增加网卡缓存来解决问题 1.ethtool -g eth0

Ring parameters for eth0: Pre-set maximums: RX: 1020 RX Mini: 0 RX Jumbo: 4080 TX: 255 Current hardware settings: RX: 255 RX Mini: 0 RX Jumbo: 0 TX: 255

2.ethtool -G eth0 rx 1000 ethtool -g eth0

Ring parameters for eth0: Pre-set maximums: RX: 1020 RX Mini: 0 RX Jumbo: 4080 TX: 255 Current hardware settings: RX: 1000 RX Mini: 0 RX Jumbo: 0 TX: 255

ifconfig

eth0 Link encap:Ethernet HWaddr 00:26:B9:3B:XX:C3 inet addr:61.255.xx.xx Bcast:61.255.xx.xx Mask:255.255.255.192 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1348879 errors:0 dropped:2714 overruns:0 frame:0 TX packets:1449564 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:646430926 (616.4 MiB) TX bytes:576842481 (550.1 MiB) Interrupt:82 Memory:da000000-da012800

ethtool -S eth0

NIC statistics: rx_bytes: 645417070 rx_error_bytes: 0 tx_bytes: 575096746 tx_error_bytes: 0 rx_ucast_packets: 1345249 rx_mcast_packets: 17 rx_bcast_packets: 826 tx_ucast_packets: 1446406 tx_mcast_packets: 0 tx_bcast_packets: 0 tx_mac_errors: 0 tx_carrier_errors: 0 rx_crc_errors: 0 rx_align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 tx_deferred: 0 tx_excess_collisions: 0 tx_late_collisions: 0 tx_total_collisions: 0 rx_fragments: 0 rx_jabbers: 0 rx_undersize_packets: 0 rx_oversize_packets: 0 rx_64_byte_packets: 620303 rx_65_to_127_byte_packets: 87492 rx_128_to_255_byte_packets: 36556 rx_256_to_511_byte_packets: 212099 rx_512_to_1023_byte_packets: 45961 rx_1024_to_1522_byte_packets: 343681 rx_1523_to_9022_byte_packets: 0 tx_64_byte_packets: 368814 tx_65_to_127_byte_packets: 714315 tx_128_to_255_byte_packets: 13048 tx_256_to_511_byte_packets: 18258 tx_512_to_1023_byte_packets: 39752 tx_1024_to_1522_byte_packets: 292219 tx_1523_to_9022_byte_packets: 0 rx_xon_frames: 0 rx_xoff_frames: 0 tx_xon_frames: 0 tx_xoff_frames: 0 rx_mac_ctrl_frames: 0 rx_filtered_packets: 0 rx_discards: 0 rx_fw_discards: 2704

rx_fw_discards: 2704 问题没有解决,依然丢包

二.更新网卡驱动 lsmod |grep bnx2

bnx2 209997 0

ethtool -i eth0

driver: bnx2 version: 1.9.3 firmware-version: 5.0.6 NCSI 2.0.3 bus-info: 0000:01:00.0

modinfo bnx2

filename: /lib/modules/2.6.18-164.el5/kernel/drivers/net/bnx2.ko version: 1.9.3 license: GPL description: Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver author: Michael Chan srcversion: D151EAED8C1037CA480DE9A alias: pci:v000014E4d0000163Csv*sd*bc*sc*i* alias: pci:v000014E4d0000163Bsv*sd*bc*sc*i* alias: pci:v000014E4d0000163Asv*sd*bc*sc*i* alias: pci:v000014E4d00001639sv*sd*bc*sc*i* alias: pci:v000014E4d000016ACsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i* alias: pci:v000014E4d0000164Csv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i* depends: vermagic: 2.6.18-164.el5 SMP mod_unload gcc-4.1 parm: disable_msi:Disable Message Signaled Interrupt (MSI) (int) parm: enable_entropy:Allow bnx2 to populate the /dev/random entropy pool (int) module_sig: 883f3504a9f766557f09578a977b7e112ebf209f6aab61295466fb787675fec1378b254df8d186609f705f1f37e06d7e2958667d258f6afe6ac1621228

1.从官网下载驱动 http://zh-cn.broadcom.com/support/ethernet_nic/netxtremeii.php

Linux iSCSI HBA only supported on RHEL 5.4, SUSE SLES 11 SP1 or newer versions of these distributions 6.2.23 03/18/11
24MB

2.安装包 unzip linux-6.2.23.zip cd Server/Linux/Driver/ rpm -ivh netxtreme2-6.2.23-1.src.rpm

1:netxtreme2 ########################################### [100%]

3.编译 cd /usr/src/redhat/ rpmbuild -bb SPECS/netxtreme2.spec

Wrote: /usr/src/redhat/RPMS/x86_64/netxtreme2-6.2.23-1.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.43778 + umask 022 + cd /usr/src/redhat/BUILD + cd netxtreme2-6.2.23 + rm -rf /var/tmp/netxtreme2-buildroot /usr/src/redhat/BUILD/file.list.netxtreme2 + exit 0

exit 0表示成功(1-255是失败)

4.安装生成的rpm包 cd RPMS/x86_64/ rpm -ivh netxtreme2-6.2.23-1.x86_64.rpm

Preparing… ########################################### [100%] 1:netxtreme2 ########################################### [100%]

新的驱动会生成在/lib/modules/2.6.18-164.el5/updates/

5.加载驱动 depmod -a modprobe bnx2 service network restart lsmod |grep bnx2

bnx2 209997 0

ethtool -i eth0

driver: bnx2 version: 1.9.3 firmware-version: 5.0.6 NCSI 2.0.3 bus-info: 0000:01:00.0

看着好像没区别,modinfo可以看到详细信息 modinfo bnx2

filename: /lib/modules/2.6.18-164.el5/updates/bnx2.ko version: 2.0.23b license: GPL description: Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver author: Michael Chan srcversion: 6E0DD070AB24C11F50B2712 alias: pci:v000014E4d0000163Csv*sd*bc*sc*i* alias: pci:v000014E4d0000163Bsv*sd*bc*sc*i* alias: pci:v000014E4d0000163Asv*sd*bc*sc*i* alias: pci:v000014E4d00001639sv*sd*bc*sc*i* alias: pci:v000014E4d000016ACsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i* alias: pci:v000014E4d0000164Csv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i* depends: vermagic: 2.6.18-164.el5 SMP mod_unload gcc-4.1 parm: disable_msi:Disable Message Signaled Interrupt (MSI) (int) parm: stop_on_tx_timeout:For debugging purposes, prevent a chip reset when a tx timeout occurs (int)

观察一段时间没有问题

20110607更新 附加进一步解决方法

在/etc/modprobe.conf中加入一行

options bnx2 disable_msi=1

创建一个脚本来重启服务 vi net.sh

/etc/init.d/network stop rmmod bnx2 modprobe bnx2 /etc/init.d/network start

chmod a+x net.sh ./net.sh

参考: http://www.pcwind.net/dell-r410.html http://bbs.linuxtone.org/thread-3813-1-1.html