freebsd 7.2 内核优化提高负载

uname -a
FreeBSD web_1 7.2-RELEASE FreeBSD 7.2-RELEASE #0: Fri May 1 07:18:07 UTC 2009 [email protected]:/usr/obj/usr/src/sys/GENERIC amd64

nginx 0.7.63作反向代理

worker_processes 16;
worker_rlimit_nofile 51200;
events {
use kqueue;
worker_connections 51200;
}

#tail /var/log/messages

Nov 25 21:47:11 web_1 kernel: kern.maxfiles limit exceeded by uid 88, please see tuning(7).
Nov 25 21:47:12 web_1 kernel: kern.maxfiles limit exceeded by uid 80, please see tuning(7).
Nov 25 21:47:44 web_1 last message repeated 27 times
Nov 25 21:48:16 web_1 last message repeated 17 times
Nov 26 09:04:01 web_1 kernel: Limiting open port RST response from 341 to 200 packets/sec

查看当前内核参数
#sysctl -a

kern.maxfiles: 12328
kern.maxfilesperproc: 11095
net.inet.icmp.icmplim: 200
net.inet.icmp.icmplim_output: 1
kern.ipc.somaxconn:128
net.inet.tcp.blackhole: 0
net.inet.udp.blackhole: 0
net.inet.udp.maxdgram: 9216
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
kern.ipc.nmbclusters: 25600
vfs.vmiodirenable: 1
net.inet.tcp.msl: 30000
net.inet.tcp.finwait2_timeout: 60000
net.inet.tcp.fast_finwait2_recycle: 0

优化内核,立即生效

#sysctl -w kern.maxfiles=65536
#sysctl -w kern.maxfilesperproc=65536
#sysctl -w net.inet.icmp.icmplim=500
#sysctl -w net.inet.icmp.icmplim_output=0
#sysctl -w kern.ipc.somaxconn=4096
#sysctl -w net.inet.tcp.blackhole=2
#sysctl -w net.inet.udp.blackhole=1
#sysctl -w net.inet.udp.maxdgram=65536
#sysctl -w net.inet.tcp.sendspace=65536
#sysctl -w net.inet.tcp.recvspace=65536
#sysctl -w kern.ipc.nmbclusters=32768
#sysctl -w net.inet.tcp.msl=2500
#sysctl -w net.inet.tcp.finwait2_timeout=600
#sysctl -w net.inet.tcp.fast_finwait2_recycle=1

以下三个参数可以在几分钟后大幅减少FIN_WAIT_2 等待数
net.inet.tcp.msl
net.inet.tcp.finwait2_timeout
net.inet.tcp.fast_finwait2_recycle

保存参数
#vi /etc/sysctl.conf

kern.maxfiles=65536
kern.maxfilesperproc=65536
net.inet.icmp.icmplim=500
net.inet.icmp.icmplim_output=0
kern.ipc.somaxconn=4096
net.inet.tcp.blackhole=2
net.inet.udp.blackhole=1
net.inet.udp.maxdgram=65536
net.inet.tcp.sendspace=65536
net.inet.tcp.recvspace=65536
kern.ipc.nmbclusters=32768
net.inet.tcp.msl=2500
net.inet.tcp.finwait2_timeout=600
net.inet.tcp.fast_finwait2_recycle=1

#vi /boot/loader.conf

kern.ipc.nmbclusters=”102400″
kern.ipc.nmbufs=”409600″

这2个参数须重启服务器才能生效

查看tcp状态
#netstat -n | awk ‘/^tcp/ {++S[$NF]} END {for(a in S) print a,S[a]}’

FIN_WAIT_1 1610
FIN_WAIT_2 19
SYN_SENT 54
LAST_ACK 286
CLOSING 7
CLOSE_WAIT 2
SYN_RCVD 55
TIME_WAIT 213
ESTABLISHED 10928

#netstat -m

2284/25241/27525 mbufs in use (current/cache/total)
664/14398/15062/32768 mbuf clusters in use (current/cache/total/max)
664/14312 mbuf+clusters out of packet secondary zone in use (current/cache)
998/8109/9107/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
5891K/67542K/73433K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
160 requests for I/O initiated by sendfile
0 calls to protocol drain routines

参考
http://wiki.freebsdchina.org/doc/s/sysctl/tuning
http://blog.csdn.net/hjue/archive/2005/12/16/553703.aspx