Dell R410 网卡在Centos5.5 上丢包

现象是每几十次访问网页就会一次打不开,数据库,临控等才时常取不到数据.
好像r410的网卡和centos5.5不怎么兼容

uname -a

Linux c1gserver 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

dmesg |grep bnx

Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.9.3 (March 17, 2009)
bnx2: eth0: using MSIX
bnx2: eth0 NIC Copper Link is Up, 100 Mbps full duplex

ifconfig
有大量的丢包;dropped:6620400

eth0 Link encap:Ethernet HWaddr 00:26:B9:3B:XX:C3
inet addr:61.255.xx.xx Bcast:61.255.xx.xx Mask:255.255.255.192
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7330597243 errors:1 dropped:6620400 overruns:0 frame:1
TX packets:8116489614 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3435468718053 (3.1 TiB) TX bytes:3656517601592 (3.3 TiB)
Interrupt:82 Memory:da000000-da012800

一.先尝试增加网卡缓存来解决问题
1.ethtool -g eth0

Ring parameters for eth0:
Pre-set maximums:
RX: 1020
RX Mini: 0
RX Jumbo: 4080
TX: 255
Current hardware settings:
RX: 255
RX Mini: 0
RX Jumbo: 0
TX: 255

2.ethtool -G eth0 rx 1000
ethtool -g eth0

Ring parameters for eth0:
Pre-set maximums:
RX: 1020
RX Mini: 0
RX Jumbo: 4080
TX: 255
Current hardware settings:
RX: 1000
RX Mini: 0
RX Jumbo: 0
TX: 255

ifconfig

eth0 Link encap:Ethernet HWaddr 00:26:B9:3B:XX:C3
inet addr:61.255.xx.xx Bcast:61.255.xx.xx Mask:255.255.255.192
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1348879 errors:0 dropped:2714 overruns:0 frame:0
TX packets:1449564 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:646430926 (616.4 MiB) TX bytes:576842481 (550.1 MiB)
Interrupt:82 Memory:da000000-da012800

ethtool -S eth0

NIC statistics:
rx_bytes: 645417070
rx_error_bytes: 0
tx_bytes: 575096746
tx_error_bytes: 0
rx_ucast_packets: 1345249
rx_mcast_packets: 17
rx_bcast_packets: 826
tx_ucast_packets: 1446406
tx_mcast_packets: 0
tx_bcast_packets: 0
tx_mac_errors: 0
tx_carrier_errors: 0
rx_crc_errors: 0
rx_align_errors: 0
tx_single_collisions: 0
tx_multi_collisions: 0
tx_deferred: 0
tx_excess_collisions: 0
tx_late_collisions: 0
tx_total_collisions: 0
rx_fragments: 0
rx_jabbers: 0
rx_undersize_packets: 0
rx_oversize_packets: 0
rx_64_byte_packets: 620303
rx_65_to_127_byte_packets: 87492
rx_128_to_255_byte_packets: 36556
rx_256_to_511_byte_packets: 212099
rx_512_to_1023_byte_packets: 45961
rx_1024_to_1522_byte_packets: 343681
rx_1523_to_9022_byte_packets: 0
tx_64_byte_packets: 368814
tx_65_to_127_byte_packets: 714315
tx_128_to_255_byte_packets: 13048
tx_256_to_511_byte_packets: 18258
tx_512_to_1023_byte_packets: 39752
tx_1024_to_1522_byte_packets: 292219
tx_1523_to_9022_byte_packets: 0
rx_xon_frames: 0
rx_xoff_frames: 0
tx_xon_frames: 0
tx_xoff_frames: 0
rx_mac_ctrl_frames: 0
rx_filtered_packets: 0
rx_discards: 0
rx_fw_discards: 2704

rx_fw_discards: 2704
问题没有解决,依然丢包

二.更新网卡驱动
lsmod |grep bnx2

bnx2 209997 0

ethtool -i eth0

driver: bnx2
version: 1.9.3
firmware-version: 5.0.6 NCSI 2.0.3
bus-info: 0000:01:00.0

modinfo bnx2

filename: /lib/modules/2.6.18-164.el5/kernel/drivers/net/bnx2.ko
version: 1.9.3
license: GPL
description: Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver
author: Michael Chan
srcversion: D151EAED8C1037CA480DE9A
alias: pci:v000014E4d0000163Csv*sd*bc*sc*i*
alias: pci:v000014E4d0000163Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000163Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001639sv*sd*bc*sc*i*
alias: pci:v000014E4d000016ACsv*sd*bc*sc*i*
alias: pci:v000014E4d000016AAsv*sd*bc*sc*i*
alias: pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i*
alias: pci:v000014E4d0000164Csv*sd*bc*sc*i*
alias: pci:v000014E4d0000164Asv*sd*bc*sc*i*
alias: pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i*
alias: pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i*
depends:
vermagic: 2.6.18-164.el5 SMP mod_unload gcc-4.1
parm: disable_msi:Disable Message Signaled Interrupt (MSI) (int)
parm: enable_entropy:Allow bnx2 to populate the /dev/random entropy pool (int)
module_sig: 883f3504a9f766557f09578a977b7e112ebf209f6aab61295466fb787675fec1378b254df8d186609f705f1f37e06d7e2958667d258f6afe6ac1621228

1.从官网下载驱动
http://zh-cn.broadcom.com/support/ethernet_nic/netxtremeii.php

Linux
iSCSI HBA only supported on RHEL 5.4, SUSE SLES 11 SP1 or newer versions of these distributions
6.2.23 03/18/11
24MB

2.安装包
unzip linux-6.2.23.zip
cd Server/Linux/Driver/
rpm -ivh netxtreme2-6.2.23-1.src.rpm

1:netxtreme2 ########################################### [100%]

3.编译
cd /usr/src/redhat/
rpmbuild -bb SPECS/netxtreme2.spec

Wrote: /usr/src/redhat/RPMS/x86_64/netxtreme2-6.2.23-1.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.43778
+ umask 022
+ cd /usr/src/redhat/BUILD
+ cd netxtreme2-6.2.23
+ rm -rf /var/tmp/netxtreme2-buildroot /usr/src/redhat/BUILD/file.list.netxtreme2
+ exit 0

exit 0表示成功(1-255是失败)

4.安装生成的rpm包
cd RPMS/x86_64/
rpm -ivh netxtreme2-6.2.23-1.x86_64.rpm

Preparing… ########################################### [100%]
1:netxtreme2 ########################################### [100%]

新的驱动会生成在/lib/modules/2.6.18-164.el5/updates/

5.加载驱动
depmod -a
modprobe bnx2
service network restart
lsmod |grep bnx2

bnx2 209997 0

ethtool -i eth0

driver: bnx2
version: 1.9.3
firmware-version: 5.0.6 NCSI 2.0.3
bus-info: 0000:01:00.0

看着好像没区别,modinfo可以看到详细信息
modinfo bnx2

filename: /lib/modules/2.6.18-164.el5/updates/bnx2.ko
version: 2.0.23b
license: GPL
description: Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver
author: Michael Chan
srcversion: 6E0DD070AB24C11F50B2712
alias: pci:v000014E4d0000163Csv*sd*bc*sc*i*
alias: pci:v000014E4d0000163Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000163Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001639sv*sd*bc*sc*i*
alias: pci:v000014E4d000016ACsv*sd*bc*sc*i*
alias: pci:v000014E4d000016AAsv*sd*bc*sc*i*
alias: pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i*
alias: pci:v000014E4d0000164Csv*sd*bc*sc*i*
alias: pci:v000014E4d0000164Asv*sd*bc*sc*i*
alias: pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i*
alias: pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i*
depends:
vermagic: 2.6.18-164.el5 SMP mod_unload gcc-4.1
parm: disable_msi:Disable Message Signaled Interrupt (MSI) (int)
parm: stop_on_tx_timeout:For debugging purposes, prevent a chip reset when a tx timeout occurs (int)

观察一段时间没有问题

20110607更新
附加进一步解决方法

在/etc/modprobe.conf中加入一行

options bnx2 disable_msi=1

创建一个脚本来重启服务
vi net.sh

/etc/init.d/network stop
rmmod bnx2
modprobe bnx2
/etc/init.d/network start

chmod a+x net.sh
./net.sh

参考:
http://www.pcwind.net/dell-r410.html
http://bbs.linuxtone.org/thread-3813-1-1.html