Skip to content

inotify+rsync+nginx+fastcgi双机负载均衡discuzx2

一.概述

服务器 web1 R410 E55202 16G SAS300G2 centos5.5 ip:192.168.1.21 web2(new) R410 E56062 16G SAS300G2 centos5.5 ip:192.168.1.23 db R410 E55042 16G SAS146G2 centos5.3

discuz论坛现有500多万贴子,近2万人在线,400万pv/日, 新增web2服务器分担论坛php处理. 尝试过在web2用nfs mount web1的程序来跑,但是访问速度太慢,无法接受 改用inotify+rsync效果不错,原来web1负载8左右,现两台各自3.5

web2安装

web2安装好php后配置php-fpm

192.168.1.23:9002 #监听地址及端口 rsyncuser#后面rsync同步的用户 website#同步用户的用户组 128#目前开128个 大概占用物理内存4G左右 192.168.1.21,192.168.1.23 #充许请求的服务器(web1,web2自已)

web2打开iptables端口

iptables -A INPUT -p tcp -m tcp -s 192.168.1.21 –dport 9002 -j ACCEPT /etc/init.d/iptables save

db服务器加权限 给web2加上和web1一样的用户访问权限(discuz,ucenter…)

web1安装设置

web1安装inotify https://github.com/rvoicilas/inotify-tools/wiki/ inotify-tools 3.14 is the latest version, released on the 7th of March 2010.

wget –no-check-certificate http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz tar zxvf inotify-tools-3.14.tar.gz cd inotify-tools-3.14 make make install 完成后，注意查看manpage，man inotify、man inotifywait

查看是否支持inotify 从kernel 2.6.13开始正式并入内核，RHEL5已经支持。 看看是否有 /proc/sys/fs/inotify/目录，以确定内核是否支持inotify ll /proc/sys/fs/inotify

total 0 -rw-r–r– 1 root root 0 Sep 14 14:01 max_queued_events -rw-r–r– 1 root root 0 Sep 14 14:01 max_user_instances -rw-r–r– 1 root root 0 Sep 14 14:01 max_user_watches

测试inotify /usr/local/bin/inotifywait -mrq –timefmt ‘%d/%m/%y %H:%M’ –format ‘%T %w%f’ -e modify,delete,create,attrib /opt/lampp/htdocs/bbs

16/09/11 15:59 /opt/lampp/htdocs/bbs/data/cache/forum_threadviews_1.log 16/09/11 15:59 /opt/lampp/htdocs/bbs/data/cache/forum_threadviews_1.log 16/09/11 15:59 /opt/lampp/htdocs/bbs/data/cache/forum_threadviews_1.log ….

rsync同步 方法一:使用ssh用户免登录 方法二:使用rsync认证免登录

我这里使用方法一

web2增加同步用户 useradd -g website rsyncuser passwd rsyncuser

web1创建公钥并传到web2 ssh-keygen -t rsa

Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is:

复制到web2

scp -P 22 ~/.ssh/id_rsa.pub [email protected]:.

web2设置免登录 将id_rsa.pub导入到.ssh/authorized_keys

cd /home/rsyncuser mkdir .ssh cat id_rsa.pub >> .ssh/authorized_keys chown -R rsyncuser:website .ssh rm id_rsa.pub

可以在web1测试下登录 ssh -p 22 [email protected]

同步脚本 inofity 的exclude支持posix正则,但只能写一个. 注意两边放bbs的路径要一样 可以先用rsync同步一下

vi inotify.sh

#!/bin/sh yesterday=`date +%Y%m%d` src=/opt/lampp/htdocs/bbs [email protected]:/opt/lampp/htdocs/ /usr/local/bin/inotifywait -mrq –exclude “data/(threadcache|log|template|sendmail\.lock|cache)” –timefmt ‘%d/%m/%y %H:%M’ –format ‘%T %w%f’ \ -e modify,delete,create,attrib ${src} | while read file do echo -e $(date +%Y-%m-%d_%H:%M:%S)”\r” >>rsynclog/inotify.${yesterday}.log rsync -av –delete –exclude “data/threadcache/” –exclude “data/log/” –exclude “data/template/” –exclude “data/sendmail.lock” ${src} -e “/usr/bin/ssh -p 22” ${des} >>rsynclog/inotify.${yesterday}.log done

在web2上创建不同步的目录和文件 cd /opt/lampp/htdocs/bbs mkdir data/{threadcache,log,template} chown -R rsyncuser:website data/{threadcache,log,template} chmod -R 0775 data/{threadcache,log,template} touch data/sendmail.lock chown -R rsyncuser:website data/sendmail.lock chmod -R 0775 data/sendmail.lock

设置权限并运行 chmod 700 ./inofity.sh ./inofity.sh &

加到开机运行

echo ‘cd /opt/shell && ./inofity.sh & ‘>>/etc/rc.local

cache目录的更新 如果使用memcache可以跳过此步 使用文件缓存时 /opt/lampp/htdocs/bbs/data/cache 目录有新注册会员及在线人数等cache,更新很频繁不适于放入inotify中 vi rsync.sh

#!/bin/sh #* * * * * cd /opt/shell && /bin/sh ./rsync.sh > /dev/null 2>&1 yesterday=`date +%Y%m%d` src=/opt/lampp/htdocs/bbs/data/cache [email protected]:/opt/lampp/htdocs/bbs/data/ echo -e $(date +%Y-%m-%d_%H:%M:%S)”\r” >>rsynclog/rsync.${yesterday}.log rsync -av –delete ${src} -e “/usr/bin/ssh -p 22” ${des2} >>rsynclog/rsync.${yesterday}.log

设置权限 chmod 775 ./rsync.sh

crontab -e 加入crontab中 每分钟运行

* * * * * cd /opt/shell && /bin/sh ./rsync.sh > /dev/null 2>&1

第一次同步完后,开启php /opt/php/sbin/php-fpm start

web1开启负载均衡

web1上调整nginx 先创建个test.php来测试,可以在phpinfo里看到不同的机器名或通firebug查看文件header来确定是否在web2上

echo ” > test.php

vi /opt/nginx/conf/nginx.conf

location ~ /test\.php?$ { #fastcgi_pass 127.0.0.1:9000; fastcgi_pass 192.168.1.23:9002; add_header App-Server php2; fastcgi_index index.php; include fcgi.conf; break; }

正式上线只需针对forum.php进行负载均衡,90%的负载都在这文件上(只开启论坛情况下)

upstream backend { ip_hash; #确保分流内网能访问公网,或无对外访问需求 server 192.168.1.23:9002 max_fails=3 fail_timeout=60s; server 127.0.0.1:9000; } location ~ /forum\.php?$ { #fastcgi_pass unix:/tmp/php-cgi.sock; #fastcgi_pass 127.0.0.1:9000; #fastcgi_pass 192.168.1.23:9002; #add_header App-Server php2; fastcgi_pass backend; fastcgi_index index.php; include fcgi.conf; break; }

fcgi.conf

fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx; fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT $document_root; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; # PHP only, required if PHP was built with –enable-force-cgi-redirect fastcgi_param REDIRECT_STATUS 200;

重启nginx reload nginx /bin/kill -HUP cat /dev/shm/nginx.pid

discuz上传附件,后台固定在web1,读贴等操作会依据用户ip分布在两台机器上.

参考: http://www.infoq.com/cn/articles/inotify-linux-file-system-event-monitoring http://hi.baidu.com/tonyty163/blog/item/3c14ca2698672a0a918f9daf.html

Posted in Discuz/Uchome/Ucenter, 网站架构.

Tagged with , , , , , .

rev="post-1248" No comments

By C1G – 2011/09/19

ACPI引起linux系统无故重启

新装机器无故重启多次。 centos6 64bit uname -a Linux Eos 2.6.32-71.el6.x86_64 #1 SMP Fri May 20 03:51:51 BST 2011 x86_64 x86_64 x86_64 GNU/Linux

硬件为Dell R410 E56062 4G1 sas300G *2

系统负载基本为0，查看日志有acpi错误。 tail -n1000 /var/log/messages |grep -i error

Sep 11 10:25:25 C1gstudio kernel: ACPI Error: No handler for Region [IPMI] (ffff88012b6b7420) [IPMI] (20090903/evregion-319) Sep 11 10:25:25 C1gstudio kernel: ACPI Error: Region IPMI(7) has no handler (20090903/exfldio-295) Sep 11 10:25:25 C1gstudio kernel: ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._GHL] (Node ffff88012b6b6470), AE_NOT_EXIST Sep 11 10:25:25 C1gstudio kernel: ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._PMC] (Node ffff88012b6b64f0), AE_NOT_EXIST Sep 11 10:25:28 C1gstudio abrtd: dbus error: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory Sep 11 10:25:28 C1gstudio abrtd: Error requesting DBus name com.redhat.abrt, possible reasons: abrt run by non-root; dbus config is incorrect; or dbus daemon needs to be restarted to reload dbus config Sep 11 10:35:51 C1gstudio kernel: ACPI Error: No handler for Region [IPMI] (ffff88012b6b7420) [IPMI] (20090903/evregion-319) Sep 11 10:35:51 C1gstudio kernel: ACPI Error: Region IPMI(7) has no handler (20090903/exfldio-295) Sep 11 10:35:51 C1gstudio kernel: ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._GHL] (Node ffff88012b6b6470), AE_NOT_EXIST Sep 11 10:35:51 C1gstudio kernel: ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._PMC] (Node ffff88012b6b64f0), AE_NOT_EXIST Sep 11 10:35:53 C1gstudio abrtd: dbus error: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory Sep 11 10:35:53 C1gstudio abrtd: Error requesting DBus name com.redhat.abrt, possible reasons: abrt run by non-root; dbus config is incorrect; or dbus daemon needs to be restarted to reload dbus config

vi /boot/grub/grub.conf 在kernel一行最后加上acpi=off noacip

kernel … acpi=off noacip

然后重启

Posted in LINUX, 技术.

Tagged with , , .

rev="post-1244" 3 comments

By C1G – 2011/09/13

php5.2.14中strtotime格式时间戳错误引起日期混乱

状况是用smarty version 2.6.5-dev格式的时间在线上服务器显示时间不对.

{$smarty.now|date_format:”%Y/%m/%d %H:%M”}

会显示成 5068/08/16 10:30

检查smarty/plugins/shared.make_timestamp.php strtotime函数在不同版本下会有不同返回结果

$time = strtotime($string);

编写代码确认 code:

echo time(); echo ‘—‘; var_dump(strtotime(time())); ?>

win php5.3.1 result: 1313465002—bool(false)

linux php5.2.14 result: 1313465026—int(96457670026)

“96457670026”?这个就是引起错误的原因.

修改下smarty/plugins/shared.make_timestamp.php 解决问题

if (is_numeric($string) && $string != -1) return $string; $time = strtotime($string); if (is_numeric($time) && $time != -1) return $time;

Posted in PHP.

Tagged with , , , .

rev="post-1239" No comments

By C1G – 2011/08/16

更新个php验证email正则

网上随处可见的验证规则会漏掉用户名中常见字符”.”,”-“,”_”; 以及后缀中可能是二级或三级域名的情况.

‘‘ ‘+’ 和 ‘?’的规则 ‘‘ 匹配0或多次 ‘+’ 匹配1或多次 ‘?’ 匹配0或1次

下面函数的验证规则为: 必需有一个不以”.”开头的字符后接0到多个字符 接上@须有一个字符开头接上0到多个包含”.”的子域名 最后接上”.”开头2到4个字符的域名后缀

function validEmail($address) { if(!preg_match(“/^[0-9a-zA-Z_-]+(\.[0-9a-zA-Z_-]+)*@[a-zA-Z0-9_-]+(\.{1}[a-zA-Z0-9_-]+)*\.{1}[a-zA-Z]{2,4}$/i”,$address)) { return False; } return True; }

Posted in PHP.

Tagged with , , .

rev="post-1237" 1 comment

By C1G – 2011/08/12

Dell R410 网卡在Centos5.5 上丢包

现象是每几十次访问网页就会一次打不开,数据库,临控等才时常取不到数据. 好像r410的网卡和centos5.5不怎么兼容

uname -a

Linux c1gserver 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

dmesg |grep bnx

Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.9.3 (March 17, 2009) bnx2: eth0: using MSIX bnx2: eth0 NIC Copper Link is Up, 100 Mbps full duplex

ifconfig 有大量的丢包;dropped:6620400

eth0 Link encap:Ethernet HWaddr 00:26:B9:3B:XX:C3 inet addr:61.255.xx.xx Bcast:61.255.xx.xx Mask:255.255.255.192 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7330597243 errors:1 dropped:6620400 overruns:0 frame:1 TX packets:8116489614 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3435468718053 (3.1 TiB) TX bytes:3656517601592 (3.3 TiB) Interrupt:82 Memory:da000000-da012800

一.先尝试增加网卡缓存来解决问题 1.ethtool -g eth0

Ring parameters for eth0: Pre-set maximums: RX: 1020 RX Mini: 0 RX Jumbo: 4080 TX: 255 Current hardware settings: RX: 255 RX Mini: 0 RX Jumbo: 0 TX: 255

2.ethtool -G eth0 rx 1000 ethtool -g eth0

Ring parameters for eth0: Pre-set maximums: RX: 1020 RX Mini: 0 RX Jumbo: 4080 TX: 255 Current hardware settings: RX: 1000 RX Mini: 0 RX Jumbo: 0 TX: 255

ifconfig

eth0 Link encap:Ethernet HWaddr 00:26:B9:3B:XX:C3 inet addr:61.255.xx.xx Bcast:61.255.xx.xx Mask:255.255.255.192 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1348879 errors:0 dropped:2714 overruns:0 frame:0 TX packets:1449564 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:646430926 (616.4 MiB) TX bytes:576842481 (550.1 MiB) Interrupt:82 Memory:da000000-da012800

ethtool -S eth0

NIC statistics: rx_bytes: 645417070 rx_error_bytes: 0 tx_bytes: 575096746 tx_error_bytes: 0 rx_ucast_packets: 1345249 rx_mcast_packets: 17 rx_bcast_packets: 826 tx_ucast_packets: 1446406 tx_mcast_packets: 0 tx_bcast_packets: 0 tx_mac_errors: 0 tx_carrier_errors: 0 rx_crc_errors: 0 rx_align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 tx_deferred: 0 tx_excess_collisions: 0 tx_late_collisions: 0 tx_total_collisions: 0 rx_fragments: 0 rx_jabbers: 0 rx_undersize_packets: 0 rx_oversize_packets: 0 rx_64_byte_packets: 620303 rx_65_to_127_byte_packets: 87492 rx_128_to_255_byte_packets: 36556 rx_256_to_511_byte_packets: 212099 rx_512_to_1023_byte_packets: 45961 rx_1024_to_1522_byte_packets: 343681 rx_1523_to_9022_byte_packets: 0 tx_64_byte_packets: 368814 tx_65_to_127_byte_packets: 714315 tx_128_to_255_byte_packets: 13048 tx_256_to_511_byte_packets: 18258 tx_512_to_1023_byte_packets: 39752 tx_1024_to_1522_byte_packets: 292219 tx_1523_to_9022_byte_packets: 0 rx_xon_frames: 0 rx_xoff_frames: 0 tx_xon_frames: 0 tx_xoff_frames: 0 rx_mac_ctrl_frames: 0 rx_filtered_packets: 0 rx_discards: 0 rx_fw_discards: 2704

rx_fw_discards: 2704 问题没有解决,依然丢包

二.更新网卡驱动 lsmod |grep bnx2

bnx2 209997 0

ethtool -i eth0

driver: bnx2 version: 1.9.3 firmware-version: 5.0.6 NCSI 2.0.3 bus-info: 0000:01:00.0

modinfo bnx2

filename: /lib/modules/2.6.18-164.el5/kernel/drivers/net/bnx2.ko version: 1.9.3 license: GPL description: Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver author: Michael Chan srcversion: D151EAED8C1037CA480DE9A alias: pci:v000014E4d0000163Csv*sd*bc*sc*i* alias: pci:v000014E4d0000163Bsv*sd*bc*sc*i* alias: pci:v000014E4d0000163Asv*sd*bc*sc*i* alias: pci:v000014E4d00001639sv*sd*bc*sc*i* alias: pci:v000014E4d000016ACsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i* alias: pci:v000014E4d0000164Csv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i* depends: vermagic: 2.6.18-164.el5 SMP mod_unload gcc-4.1 parm: disable_msi:Disable Message Signaled Interrupt (MSI) (int) parm: enable_entropy:Allow bnx2 to populate the /dev/random entropy pool (int) module_sig: 883f3504a9f766557f09578a977b7e112ebf209f6aab61295466fb787675fec1378b254df8d186609f705f1f37e06d7e2958667d258f6afe6ac1621228

1.从官网下载驱动 http://zh-cn.broadcom.com/support/ethernet_nic/netxtremeii.php

Linux iSCSI HBA only supported on RHEL 5.4, SUSE SLES 11 SP1 or newer versions of these distributions 6.2.23 03/18/11
24MB

2.安装包 unzip linux-6.2.23.zip cd Server/Linux/Driver/ rpm -ivh netxtreme2-6.2.23-1.src.rpm

1:netxtreme2 ########################################### [100%]

3.编译 cd /usr/src/redhat/ rpmbuild -bb SPECS/netxtreme2.spec

Wrote: /usr/src/redhat/RPMS/x86_64/netxtreme2-6.2.23-1.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.43778 + umask 022 + cd /usr/src/redhat/BUILD + cd netxtreme2-6.2.23 + rm -rf /var/tmp/netxtreme2-buildroot /usr/src/redhat/BUILD/file.list.netxtreme2 + exit 0

exit 0表示成功(1-255是失败)

4.安装生成的rpm包 cd RPMS/x86_64/ rpm -ivh netxtreme2-6.2.23-1.x86_64.rpm

Preparing… ########################################### [100%] 1:netxtreme2 ########################################### [100%]

新的驱动会生成在/lib/modules/2.6.18-164.el5/updates/

5.加载驱动 depmod -a modprobe bnx2 service network restart lsmod |grep bnx2

bnx2 209997 0

ethtool -i eth0

driver: bnx2 version: 1.9.3 firmware-version: 5.0.6 NCSI 2.0.3 bus-info: 0000:01:00.0

看着好像没区别,modinfo可以看到详细信息 modinfo bnx2

filename: /lib/modules/2.6.18-164.el5/updates/bnx2.ko version: 2.0.23b license: GPL description: Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver author: Michael Chan srcversion: 6E0DD070AB24C11F50B2712 alias: pci:v000014E4d0000163Csv*sd*bc*sc*i* alias: pci:v000014E4d0000163Bsv*sd*bc*sc*i* alias: pci:v000014E4d0000163Asv*sd*bc*sc*i* alias: pci:v000014E4d00001639sv*sd*bc*sc*i* alias: pci:v000014E4d000016ACsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv*sd*bc*sc*i* alias: pci:v000014E4d000016AAsv0000103Csd00003102bc*sc*i* alias: pci:v000014E4d0000164Csv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv*sd*bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003106bc*sc*i* alias: pci:v000014E4d0000164Asv0000103Csd00003101bc*sc*i* depends: vermagic: 2.6.18-164.el5 SMP mod_unload gcc-4.1 parm: disable_msi:Disable Message Signaled Interrupt (MSI) (int) parm: stop_on_tx_timeout:For debugging purposes, prevent a chip reset when a tx timeout occurs (int)

观察一段时间没有问题

20110607更新 附加进一步解决方法

在/etc/modprobe.conf中加入一行

options bnx2 disable_msi=1

创建一个脚本来重启服务 vi net.sh

/etc/init.d/network stop rmmod bnx2 modprobe bnx2 /etc/init.d/network start

chmod a+x net.sh ./net.sh

参考: http://www.pcwind.net/dell-r410.html http://bbs.linuxtone.org/thread-3813-1-1.html

Posted in linux 维护优化, 技术.

Tagged with , , , .

rev="post-1230" No comments

By C1G – 2011/04/28

Lempelf一键包for rhel/centos 64bit发布

Lempelf一键安装包是什么?

Lempelf一键安装包是用Shell编写的在Linux平台快速安装nginx+php+mysql的Shell程序。

我们为什么需要它?

编译安装需要输入大量的命令，如果是配置生产环境需要耗费大量的时间。 不会Linux的站长或Linux新手想使用Linux作为生产环境……

它有什么优势?

无需一个一个的输入命令，无需值守，编译安装优化编译参数，提高性能，解决不必要的软件间依赖。

介绍及下载:https://blog.c1gstudio.com/lempelfpage

Posted in Lempelf一键包, shell, 技术.

Tagged with .

rev="post-1228" No comments

By C1G – 2011/04/27

把xen虚拟机映像文件转化成基于lvm分区的

把xen虚拟机映像文件转化成基于lvm分区的

1.查看当前有两块硬盘 fdisk -l

Disk /dev/sda: 146.8 GB, 146815733760 bytes 255 heads, 63 sectors/track, 17849 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 25 200781 83 Linux /dev/sda2 26 17849 143171280 8e Linux LVM Disk /dev/sdb: 146.8 GB, 146815733760 bytes 255 heads, 63 sectors/track, 17849 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 * 1 17849 143372061 8e Linux LVM

2.查看lmv组 vgscan

Reading all physical volumes. This may take a while… Found volume group “VolGroup00” using metadata type lvm2

只有一个组

3.查看lv分区 lvscan

ACTIVE ‘/dev/VolGroup00/LogVol02’ [30.00 GB] inherit ACTIVE ‘/dev/VolGroup00/LogVol03’ [92.53 GB] inherit ACTIVE ‘/dev/VolGroup00/LogVol01’ [10.00 GB] inherit ACTIVE ‘/dev/VolGroup00/LogVol00’ [4.00 GB] inherit

4.查看pv情况 pvscan

PV /dev/sda2 VG VolGroup00 lvm2 [136.53 GB / 0 free] Total: 1 [136.53 GB] / in use: 1 [136.53 GB] / in no VG: 0 [0 ]

硬盘/dev/sda2 分配给VolGroup00组,第二块硬盘还未分配

5.给第二块硬盘创建pv pvcreate /dev/sdb1

Physical volume “/dev/sdb1” successfully created

6.再次查看pv,已分配 pvscan

PV /dev/sda2 VG VolGroup00 lvm2 [136.53 GB / 0 free] PV /dev/sdb1 lvm2 [136.73 GB] Total: 2 [273.26 GB] / in use: 1 [136.53 GB] / in no VG: 1 [136.73 GB]

7.创建新卷组 vgcreate VolGroup01 /dev/sdb1

Volume group “VolGroup01” successfully created

8.激活新卷组 vgchange -a y VolGroup01

0 logical volume(s) in volume group “VolGroup01” now active

9.查看vm1虚拟机映像文件大小 ls -lh /opt/vm1/vm1.img

-rw-r–r– 1 root root 15G Apr 12 15:32 /opt/vm1/vm1.img

10.创建一个和vm1虚拟机映像文件一样大小的lvm分区 lvcreate -L15G -n Vol01 VolGroup01

Logical volume “Vol01” created

11.关闭虚拟机 xm shutdown vm1

12.复制虚拟机映像文件和配置文件 dd if=/opt/vm1/vm1.img of=/dev/VolGroup01/Vol01

30722048+0 records in 30722048+0 records out 15729688576 bytes (16 GB) copied, 646.077 seconds, 24.3 MB/s

\cp /etc/xen/vm1 /etc/xen/vm3

13.修改虚拟机配置文件 vi /etc/xen/vm3

name = “vm1” uuid = “85386e79-9f79-e243-9b62-3c9da736ae9f” maxmem = 2048 memory = 2048 vcpus = 2 bootloader = “/usr/bin/pygrub” on_poweroff = “destroy” on_reboot = “restart” on_crash = “restart” disk = [ “tap:aio:/opt/vm1/vm1.img,xvda,w” ] vif = [ “mac=00:16:36:47:a2:89,bridge=virbr0,script=vif-bridge” ]

修改name,uuid,mac保持唯一,通常将最后一位加1,修改disk路径. 修改后的配置

name = “vm3” uuid = “85386e79-9f79-e243-9b62-3c9da736ae1f” maxmem = 2048 memory = 2048 vcpus = 2 bootloader = “/usr/bin/pygrub” on_poweroff = “destroy” on_reboot = “restart” on_crash = “restart” disk = [ “phy:/dev/VolGroup01/Vol01,xvda,w” ] vif = [ “mac=00:16:36:47:a2:81,bridge=virbr0,script=vif-bridge” ]

14.启动并进入虚拟机 vm create vm3 -c

15.修改主机名 15.1 hostname vm3

15.2 vi /etc/sysconfig/network

15.3 vi /etc/hosts

16.修改网卡ip和mac,和xen配置对应 vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0 BOOTPROTO=static BROADCAST=192.168.122.255 HWADDR=00:16:36:47:A2:81 IPADDR=192.168.122.13 NETMASK=255.255.255.0 NETWORK=192.168.122.0 ONBOOT=yes

17.重启网络,完成修改 /etc/init.d/network restart

18.测试 18.1 ctrl+]退出虚拟机 18.2启动”vm1″虚拟机 18.3测试vm1网络 ping 192.168.122.11 18.4测试vm3网络 ping 192.168.122.13 18.5测试虚拟机之间的网络 vm console vm3 ping 192.168.122.11

ps:随宿主启动及服务相关iptables不要忘记

参考:https://blog.c1gstudio.com/archives/1215

Posted in 技术, 虚拟化/xen.

Tagged with , .

rev="post-1218" No comments

By C1G – 2011/04/12

XEN虚拟机复制

以映像文件方式保存的虚拟机复制非常方便. 只要拷贝xen vm的img镜像文件和配置文件，并修改相应配置文就可以了

1.关闭虚拟机 xm shutdown vm1

2.复制虚拟机映像文件和配置文件 \cp /opt/vm1/vm1.img /opt/vm1/vm2.img \cp /etc/xen/vm1 /etc/xen/vm2

3.修改虚拟机配置文件 vi /etc/xen/vm2

name = “vm1” uuid = “85386e79-9f79-e243-9b62-3c9da736ae9f” maxmem = 2048 memory = 2048 vcpus = 2 bootloader = “/usr/bin/pygrub” on_poweroff = “destroy” on_reboot = “restart” on_crash = “restart” disk = [ “tap:aio:/opt/vm1/vm1.img,xvda,w” ] vif = [ “mac=00:16:36:47:a2:89,bridge=virbr0,script=vif-bridge” ]

修改name,uuid,mac保持唯一,通常将最后一位加1,修改disk路径. 修改后的配置

name = “vm2” uuid = “85386e79-9f79-e243-9b62-3c9da736ae0f” maxmem = 2048 memory = 2048 vcpus = 2 bootloader = “/usr/bin/pygrub” on_poweroff = “destroy” on_reboot = “restart” on_crash = “restart” disk = [ “tap:aio:/opt/vm1/vm2.img,xvda,w” ] vif = [ “mac=00:16:36:47:a2:80,bridge=virbr0,script=vif-bridge” ]

4.启动并进入虚拟机 vm create vm2 -c

5.修改主机名 5.1 hostname vm2

5.2 vi /etc/sysconfig/network

5.3 vi /etc/hosts

6.修改网卡ip和mac,和xen配置对应 vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0 BOOTPROTO=static BROADCAST=192.168.122.255 HWADDR=00:16:36:47:A2:80 IPADDR=192.168.122.12 NETMASK=255.255.255.0 NETWORK=192.168.122.0 ONBOOT=yes

7.重启网络,完成修改 /etc/init.d/network restart

8.测试 8.1 ctrl+]退出虚拟机 8.2启动”vm1″虚拟机 8.3测试vm1网络 ping 192.168.122.11 8.4测试vm2网络 ping 192.168.122.12 8.5测试虚拟机之间的网络 vm console vm2 ping 192.168.122.11

ps:随宿主启动及服务相关iptables不要忘记

Posted in 技术, 虚拟化/xen.

Tagged with , .

rev="post-1215" 1 comment

By C1G – 2011/04/08

Linux下用zip给压缩包加上密码

1.打包tmp目录为file.tar tar -cf file.tar tmp/

2.给file.tar文件加上密码123456 并删除原始文件 zip -P 123456 -m file.zip file.tar

adding: file.zip (deflated 96%)

3.解压file.zip unzip file.zip

Archive: file.zip [file.zip] file password:

zip功能说明：压缩文件。 unzip功能说明：解压缩zip文件

语　　法：zip [-AcdDfFghjJKlLmoqrSTuvVwXyz$][-b ][-ll][-n ][-t ][-][压缩文件][文件…][-i ][-x ]

补充说明：zip是个使用广泛的压缩程序，文件经它压缩后会另外产生具有”.zip”扩展名的压缩文件。

参　　数： -A 调整可执行的自动解压缩文件。 -b 指定暂时存放文件的目录。

-c 替每个被压缩的文件加上注释。 -d 从压缩文件内删除指定的文件。 -D 压缩文件内不建立目录名称。 -f 此参数的效果和指定”-u”参数类似，但不仅更新既有文件，如果某些文件原本不存在于压缩文件内，使用本参数会一并将其加入压缩文件中。 -F 尝试修复已损坏的压缩文件。 -g 将文件压缩后附加在既有的压缩文件之后，而非另行建立新的压缩文件。 -h 在线帮助。 -i 只压缩符合条件的文件。 -j 只保存文件名称及其内容，而不存放任何目录名称。 -J 删除压缩文件前面不必要的数据。 -k 使用MS-DOS兼容格式的文件名称。 -l 压缩文件时，把LF字符置换成LF+CR字符。 -ll 压缩文件时，把LF+CR字符置换成LF字符。 -L 显示版权信息。 -m 将文件压缩并加入压缩文件后，删除原始文件，即把文件移到压缩文件中。 -n 不压缩具有特定字尾字符串的文件。 -o 以压缩文件内拥有最新更改时间的文件为准，将压缩文件的更改时间设成和该文件相同。 -q 不显示指令执行过程。 -r 递归处理，将指定目录下的所有文件和子目录一并处理。 -S 包含系统和隐藏文件。 -t 把压缩文件的日期设成指定的日期。 -T 检查备份文件内的每个文件是否正确无误。 -u 更换较新的文件到压缩文件内。 -v 显示指令执行过程或显示版本信息。 -V 保存VMS操作系统的文件属性。 -w 在文件名称里假如版本编号，本参数仅在VMS操作系统下有效。 -x 压缩时排除符合条件的文件。 -X 不保存额外的文件属性。 -y 直接保存符号连接，而非该连接所指向的文件，本参数仅在UNIX之类的系统下有效。 -z 替压缩文件加上注释。 -$ 保存第一个被压缩文件所在磁盘的卷册名称。 – 压缩效率是一个介于1-9的数值。

Posted in Linux 命令, 技术.

Tagged with .

rev="post-1210" No comments

By C1G – 2011/04/02

Centos5.5使用xen 3.0.3安装Linux虚拟机

网上一些中文资料不够详细,一直没装成功,经历数十次的失败现成功安装 通过桥接和端口转发,宿主机和虚拟机都可对外服务.

一.安装xen 1.系统信息 dell r410 55062,4G4,SAS146G*2

cat /etc/issue CentOS release 5.5 (Final)

uname -a Linux beetel 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

机器名:server_dom0 公网ip:61.xxx.xx.xx 虚拟机ip:192.168.122.11

查看虚似化支持 grep -E ‘(vmx|svm)’ /proc/cpuinfo

flags : fpu tsc msr pae cx8 apic mtrr cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc pni vmx est ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm

2.设置网易源更新组件 xen安装方式有yum,二进制,编译 我选择yum方式最方便 cd /etc/yum.repos.d wget http://mirrors.163.com/.help/CentOS-Base-163.repo yum makecache yum groupinstall “Development Libraries” yum groupinstall “Development Tools” yum install transfig wget texi2html libaio-devel dev86 glibc-devel e2fsprogs-devel gitk mkinitrd iasl xz-devel bzip2-devel pciutils-libs pciutils-devel SDL-devel libX11-devel gtk2-devel bridge-utils PyXML qemu-common qemu-img mercurial

2.安装centos5.5自带的xen 3.0.3-105.el5_5.5 这个是07年版的,最新为xen3.4.3 yum groupinstall Virtualization

================================================================================ Package Arch Version Repository Size ================================================================================ Installing: gnome-applet-vm x86_64 0.1.2-1.el5 base 76 k kernel-xen x86_64 2.6.18-194.32.1.el5 updates 20 M libvirt i386 0.6.3-33.el5_5.3 updates 2.0 M libvirt x86_64 0.6.3-33.el5_5.3 updates 2.0 M virt-manager x86_64 0.6.1-12.el5 base 1.5 M virt-viewer x86_64 0.0.2-3.el5 base 25 k xen x86_64 3.0.3-105.el5_5.5 updates 1.9 M Installing for dependencies: libvirt-python x86_64 0.6.3-33.el5_5.3 updates 137 k python-virtinst noarch 0.400.3-9.el5_5.1 updates 380 k Transaction Summary ================================================================================ Install 9 Package(s) Upgrade 0 Package(s) Total download size: 28 M Is this ok [y/N]: y

3.修改其中的default=1为default=0，启用xen内核 vi /etc/grub.conf

default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title CentOS (2.6.18-194.32.1.el5xen) root (hd0,0) kernel /xen.gz-2.6.18-194.32.1.el5 module /vmlinuz-2.6.18-194.32.1.el5xen ro root=/dev/VolGroup00/LogVol02 module /initrd-2.6.18-194.32.1.el5xen.img title CentOS (2.6.18-194.el5) root (hd0,0) kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/LogVol02 initrd /initrd-2.6.18-194.el5.img

4.重新启动计算机 reboot

5.重启后登录系统，检查XEN是否正常

5.1 内核版本 uname -a Linux beetel 2.6.18-194.32.1.el5xen #1 SMP Wed Jan 5 18:44:24 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

5.2 xen信息 xm info

host : server_dom0 release : 2.6.18-194.32.1.el5xen version : #1 SMP Wed Jan 5 18:44:24 EST 2011 machine : x86_64 nr_cpus : 8 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 4 threads_per_core : 1 cpu_mhz : 2128 hw_caps : bfebfbff:28100800:00000000:00000140:009ce3bd:00000000:00000001 total_memory : 16371 free_memory : 383 node_to_cpu : node0:0-7 xen_major : 3 xen_minor : 1 xen_extra : .2-194.32.1.el5 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) cc_compile_by : mockbuild cc_compile_domain : centos.org cc_compile_date : Wed Jan 5 17:43:03 EST 2011 xend_config_format : 2

5.3查看xen日志 ls -lh /var/log/xen

5.4查看网卡 ifconfig

eth0 Link encap:Ethernet HWaddr 7x:2x:Cx:0x:5x:Cx inet addr:61.xxx.xx.xx Bcast:61.xxx.xx.xx Mask:255.255.255.128 inet6 addr: fe80::xxxxxxxxxxxxxx/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:22251 errors:0 dropped:0 overruns:0 frame:0 TX packets:10210 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:20696752 (19.7 MiB) TX bytes:796183 (777.5 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:23591 errors:0 dropped:0 overruns:0 frame:0 TX packets:10233 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:21590086 (20.5 MiB) TX bytes:876169 (855.6 KiB) Interrupt:25 Memory:da000000-da012800 vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:10212 errors:0 dropped:0 overruns:0 frame:0 TX packets:22251 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:796563 (777.8 KiB) TX bytes:20696752 (19.7 MiB) virbr0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:468 (468.0 b) xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:5505 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:273968 (267.5 KiB) TX bytes:0 (0.0 b)

新增了peth0,vif0.0,virbr0,xenbr0 peth0为物理网卡 eth0为宿主机(dom0)虚拟网络设备 vif0.0第x个虚拟机(domu)中的第x块网卡,这里为dom0的接口 virbr0和xenbr0为软桥接接口

brctl show

bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes xenbr0 8000.feffffffffff no peth0 vif0.0

5.5查看宿主iptables cat /etc/sysconfig/iptables

# Generated by iptables-save v1.3.5 on Thu Mar 31 01:08:25 2011 *filter :INPUT DROP [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -i lo -j ACCEPT -A INPUT -s 127.0.0.0/255.0.0.0 -i ! lo -j DROP -A INPUT -m state –state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -p tcp -m tcp –dport 80 -j ACCEPT -A INPUT -p tcp -m tcp –dport 3306 -j ACCEPT -A INPUT -p tcp -m tcp –dport 22 -j ACCEPT -A INPUT -p udp -m udp –dport 123 -j ACCEPT -A OUTPUT -o lo -j ACCEPT -A OUTPUT -o eth0 -j ACCEPT COMMIT # Completed on Thu Mar 31 01:08:25 2011 # Generated by iptables-save v1.3.5 on Thu Mar 31 01:08:25 2011 *nat :PREROUTING ACCEPT [166:7018] :POSTROUTING ACCEPT [1:80] :OUTPUT ACCEPT [1:80] COMMIT # Completed on Thu Mar 31 01:08:25 2011 # Generated by iptables-save v1.3.5 on Thu Mar 31 01:08:25 2011 *mangle :PREROUTING ACCEPT [224:12218] :INPUT ACCEPT [58:5200] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [47:6304] :POSTROUTING ACCEPT [47:6304] COMMIT # Completed on Thu Mar 31 01:08:25 2011

二、安装XEN的Linux客户机系统

准备好安装源 1.从网易下载centos5.5的bt http://mirrors.163.com/centos/5.5/isos/x86_64/CentOS-5.5-x86_64-bin-DVD.torrent

2.刻盘

3.从cdrom 复制源到硬盘 mkdir /mnt/cdrom mkdir /opt/iso dd if=/dev/cdrom of=/opt/iso/centos.iso ll -h /opt/iso/centos.iso mount -o loop -t iso9660 /opt/iso/centos.iso /mnt/cdrom ll /mnt/cdrom

4.制作映像文件 xen能安装和运行在实际的物理分区上，lvm分区上，一个映像文件中，或NFS等网络文件系统中

映像文件安全性和io性能比较低,但是很方便. mkdir /opt/vm1 cd /opt/vm1 dd if=/dev/zero of=vm1.img bs=1M seek=15000 count=1

1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.002058 seconds, 510 MB/s

#这里创建15G大小

5.安装源获取方式

有http,ftp,nfs等方试

直接使用在线源
http://mirrors.163.com/centos/5.5/os/x86_64/
在设置完IP后就没有反应了

python -m SimpleHTTPServer
在setuptool安装时失败

使用nfs方式 vi /etc/exports /mnt/cdrom *(sync,ro)

yum install nfs-utils portmap nfs会使用随机端口,先把iptables关了 /etc/init.d/portmap start /etc/init.d/nfs start exportfs -rv

6.查看iptables iptables-save

# Generated by iptables-save v1.3.5 on Thu Mar 31 01:15:31 2011 *filter :INPUT DROP [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -i virbr0 -p udp -m udp –dport 53 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp –dport 53 -j ACCEPT -A INPUT -i virbr0 -p udp -m udp –dport 67 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp –dport 67 -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -s 127.0.0.0/255.0.0.0 -i ! lo -j DROP -A INPUT -m state –state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -p tcp -m tcp –dport 80 -j ACCEPT -A INPUT -p tcp -m tcp –dport 3306 -j ACCEPT -A INPUT -p tcp -m tcp –dport 22 -j ACCEPT -A INPUT -p udp -m udp –dport 123 -j ACCEPT -A FORWARD -d 192.168.122.0/255.255.255.0 -o virbr0 -m state –state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -s 192.168.122.0/255.255.255.0 -i virbr0 -j ACCEPT -A FORWARD -i virbr0 -o virbr0 -j ACCEPT -A FORWARD -o virbr0 -j REJECT –reject-with icmp-port-unreachable -A FORWARD -i virbr0 -j REJECT –reject-with icmp-port-unreachable -A OUTPUT -o lo -j ACCEPT -A OUTPUT -o eth0 -j ACCEPT COMMIT # Completed on Thu Mar 31 01:15:31 2011 # Generated by iptables-save v1.3.5 on Thu Mar 31 01:15:31 2011 *nat :PREROUTING ACCEPT [602:26675] :POSTROUTING ACCEPT [1:73] :OUTPUT ACCEPT [1:73] -A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p tcp -j MASQUERADE –to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p udp -j MASQUERADE –to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -j MASQUERADE COMMIT # Completed on Thu Mar 31 01:15:31 2011 # Generated by iptables-save v1.3.5 on Thu Mar 31 01:15:31 2011 *mangle :PREROUTING ACCEPT [717:40264] :INPUT ACCEPT [147:16162] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [129:19329] :POSTROUTING ACCEPT [129:19329] COMMIT # Completed on Thu Mar 31 01:15:31 2011

系统重启后xen会再生成一次,会有重复,不作保存直接关了. /etc/init.d/iptables stop

7.安装虚拟机

virt-install -n vm1 -r 2048 –vcpus=2 –file=/opt/vm1/vm1.img –nographics -p –location=nfs:192.168.122.1:/mnt/cdrom –bridge=virbr0

创建名为”vm1″的虚拟机,分配2G内存,2个cpu,使用”/opt/vm1/vm1.img”映像文件,半虚似化,使用nfs源,使用virbr0网桥 192.168.122.1为宿主virbr0的ip 我装了好N次卡在获取hostname那里,带上–bridge=virbr0参数就可以顺利通过

8.安装中 ■语言选择english ■手动配置ipv4;ip:192.168.122.11/255.255.255.0,Gateway:192.168.122.1,Name Server: 192.168.122.1 如果virt-install的网络配置错误,会一直卡在这里 ■下一步是选择”Use text mode”还是”Start VNC”,使用文本模式 ■分区

/dev/xvda ^ | | xvda1 1 13 101M ext3 /boot : | | xvda2 14 144 1027M swap : | | xvda3 145 1912 13868M ext3 / : |

■Use GRUB Boot Loader,4个ok ■配置网卡ip

IP Address Prefix (Netmask) | | 192.168.122.11__ / 255.255.255.0___

■网关和dns

| Gateway: 192.168.122.1____________________________ | | Primary DNS: 192.168.122.1____________________________ | | Secondary DNS: 8.8.8.8__________________________________ |

■机器名设置:vm1 ■时区设置:不使用utc,Asia/Shanghai ■root密码: ■安装包选择:去掉上面的*,使用自定义 选择Administration Tools,Base,Development Tools ,Editors,Text-based Internet ■依赖关系检查后按”OK”,系统开始格式化文件系统并复制文件 ■重启

三、配置宿主机及虚拟机 1.进入虚拟机 重启后在sendmail和sm-client会卡一会 出来authconfig-tui的乱码配置界面,不停的闪啊闪,键盘也不听控制,不用操作,等它闪个一分钟就会自动关了

CentOS release 5.5 (Final) Kernel 2.6.18-194.el5xen on an x86_64 vm1 login: root

输入root和密码进去后没有光标

退出并新开一个控制台然后重新进入 在宿主机 xm console vm1

2.虚拟机网卡配置 eth0 Link encap:Ethernet HWaddr 00:16:36:47:A2:89
inet addr:192.168.122.11 Bcast:192.168.122.255 Mask:255.255.255.0 inet6 addr: fe80::216:36ff:fe47:a289/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:354 errors:0 dropped:0 overruns:0 frame:0 TX packets:59 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:18489 (18.0 KiB) TX bytes:7066 (6.9 KiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:8 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:560 (560.0 b) TX bytes:560 (560.0 b)

3.虚拟机设置

3.1执行firstboot解决authconfig-tui乱码问题

3.2关闭服务方便测试 chkconfig sendmail off chkconfig iptables off

4.测试虚拟机网络 ping 61.192.168.122.1 通 ping 61.xxx.xx.xx 通 ping 8.8.8.8 不通

因为宿主机刚才安装时关了iptables,所以不通

5.宿主机设置

5.1重启服务器 先reboot,xen会自动生成iptables

5.2启动虚拟机 xm create vm1 -c

6.虚传机安装http服务 6.1测试外网 ping 8.8.8.8 通

6.2安装apache cd /etc/yum.repos.d wget http://mirrors.163.com/.help/CentOS-Base-163.repo yum makecache yum -y install httpd

6.3生成个测试首页 echo ‘vm1’>/var/www/html/index.html service httpd start chkconfig httpd on

6.4测试本机 wget 192.168.122.11

–2011-03-31 11:54:23– http://192.168.122.11/ Connecting to 192.168.122.11:80… connected. HTTP request sent, awaiting response… 200 OK Length: 4 [text/html] Saving to: `index.html.1′ 100%[======================================>] 4 –.-K/s in 0s 2011-03-31 11:54:23 (217 KB/s) – `index.html.1′ saved [4/4]

cat index.html.1 vm1

7.宿主机测试 7.1内存分配了2G给虚传机 cat /proc/meminfo

MemTotal: 14319616 kB MemFree: 13720696 kB

7.2新增了个网卡

vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:158 errors:0 dropped:0 overruns:0 frame:0 TX packets:1958 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:32 RX bytes:9679 (9.4 KiB) TX bytes:114685 (111.9 KiB)

7.3虚拟机配置文件 cat /etc/xen/vm1

name = “vm1” uuid = “85386e79-9f79-e243-9b62-3c9da736ae9f” maxmem = 2048 memory = 2048 vcpus = 2 bootloader = “/usr/bin/pygrub” on_poweroff = “destroy” on_reboot = “restart” on_crash = “restart” disk = [ “tap:aio:/opt/vm1/vm1.img,xvda,w” ] vif = [ “mac=00:16:36:47:a2:89,bridge=virbr0,script=vif-bridge” ]

7.4xend配置文件 cat /etc/xen/xend-config.sxp |grep -v “^#”|grep -v ^$

(xend-unix-server yes) (xend-unix-path /var/lib/xend/xend-socket) (xend-relocation-hosts-allow ‘^localhost$ ^localhost\\.localdomain$’) (network-script network-bridge) (vif-script vif-bridge) (dom0-min-mem 256) (dom0-cpus 0) (vncpasswd ”) (keymap ‘en-us’)

7.5虚传机列表 xm list

Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 13984 8 r—– 40.1 vm1 1 2047 2 -b—- 5.5

7.6 测试80端口http服务 wget 192.168.122.11

2011-03-31 11:55:17 (178 KB/s) – `index.html’ saved [4/4]

也没有问题

8.客户端测试 在本地电脑浏览器中输入61.xxx.xx.xx 无法连接

9.宿主机设置 9.1打开ip转发 echo ‘1’ > /proc/sys/net/ipv4/ip_forward vi /etc/sysctl.conf

# Controls IP packet forwarding net.ipv4.ip_forward = 1

9.2转发公网80端口至虚传机80端口 公网ip:61.xxx.xx.xx eht0为公网网卡 虚传机ip:192.168.122.11

iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 80 -j DNAT –to 192.168.122.11:80 下面可加可不加 iptables -A POSTROUTING -t nat -d 192.168.122.11 -p tcp -m tcp –dport 80 -j SNAT –to 61.xxx.xx.xx

9.3增加forward充许 iptables -nL

Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all — 0.0.0.0/0 192.168.122.0/24 state RELATED,ESTABLISHED ACCEPT all — 192.168.122.0/24 0.0.0.0/0 ACCEPT all — 0.0.0.0/0 0.0.0.0/0 REJECT all — 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable REJECT all — 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable ACCEPT all — 0.0.0.0/0 0.0.0.0/0 PHYSDEV match –physdev-in vif1.0 ACCEPT all — 0.0.0.0/0 0.0.0.0/0 PHYSDEV match –physdev-in vif2.0

reject-with icmp-port-unreachable这两条规则禁止了方问 解决方式一: 执行下面命令删除规则(不推荐)

iptables -D FORWARD 4 iptables -D FORWARD 4

解决方式二: 增加新的充许规则

iptables -I FORWARD -i eth0 -o virbr0 -p tcp -m state –state NEW -j ACCEPT

或

iptables -I FORWARD -o virbr0 -j ACCEPT

宿主最后的iptables

# Generated by iptables-save v1.3.5 on Thu Mar 31 15:33:44 2011 *filter :INPUT DROP [72:2903] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [24:2973] -A INPUT -i virbr0 -p udp -m udp –dport 53 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp –dport 53 -j ACCEPT -A INPUT -i virbr0 -p udp -m udp –dport 67 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp –dport 67 -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -s 127.0.0.0/255.0.0.0 -i ! lo -j DROP -A INPUT -m state –state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -p tcp -m tcp –dport 80 -j ACCEPT -A INPUT -p tcp -m tcp –dport 3306 -j ACCEPT -A INPUT -p tcp -m tcp –dport 22 -j ACCEPT -A INPUT -p udp -m udp –dport 123 -j ACCEPT -A FORWARD -i eth0 -o virbr0 -p tcp -m state –state NEW -j ACCEPT -A FORWARD -d 192.168.122.0/255.255.255.0 -o virbr0 -m state –state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -s 192.168.122.0/255.255.255.0 -i virbr0 -j ACCEPT -A FORWARD -i virbr0 -o virbr0 -j ACCEPT -A FORWARD -o virbr0 -j REJECT –reject-with icmp-port-unreachable -A FORWARD -i virbr0 -j REJECT –reject-with icmp-port-unreachable -A FORWARD -i eth0 -o virbr0 -p tcp -m state –state NEW -j ACCEPT -A FORWARD -m physdev –physdev-in vif1.0 -j ACCEPT -A OUTPUT -o lo -j ACCEPT -A OUTPUT -o eth0 -j ACCEPT COMMIT # Completed on Thu Mar 31 15:33:44 2011 # Generated by iptables-save v1.3.5 on Thu Mar 31 15:33:44 2011 *nat :PREROUTING ACCEPT [15745:680363] :POSTROUTING ACCEPT [195:14508] :OUTPUT ACCEPT [191:14292] -A PREROUTING -i eth0 -p tcp -m tcp –dport 80 -j DNAT –to-destination 192.168.122.11:80 -A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p tcp -j MASQUERADE –to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p udp -j MASQUERADE –to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -j MASQUERADE COMMIT # Completed on Thu Mar 31 15:33:44 2011 # Generated by iptables-save v1.3.5 on Thu Mar 31 15:33:44 2011 *mangle :PREROUTING ACCEPT [51572:51647208] :INPUT ACCEPT [35843:50960353] :FORWARD ACCEPT [214:22186] :OUTPUT ACCEPT [31200:2591886] :POSTROUTING ACCEPT [31414:2614072] COMMIT # Completed on Thu Mar 31 15:33:44 2011

客户端再次测试成功访问

9.4开机后添加iptables xen的iptables规则是在开机后添加到现有的iptables中,其中有一条是清空FORWARD链,所以要重新再加一次 echo ‘iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 80 -j DNAT –to 192.168.122.11:80’ >> /etc/rc.local echo ‘iptables -I FORWARD -i eth0 -o virbr0 -p tcp -m state –state NEW -j ACCEPT’ >> /etc/rc.local

vi /opt/shell/vm_iptables.sh

#!/bin/sh IPTABLES=/sbin/iptables KERNEL=`/bin/uname -r` if [ ${KERNEL: -3} = “xen” ] then #vm1 web $IPTABLES -A PREROUTING -t nat -p tcp -i eth0 –dport 80 -j DNAT –to 192.168.122.11:80 #vm1 ssh $IPTABLES -A INPUT -p tcp -m tcp –dport 7022 -j ACCEPT $IPTABLES -A PREROUTING -t nat -p tcp -i eth0 –dport 7022 -j DNAT –to 192.168.122.11:22 $IPTABLES -I FORWARD -i eth0 -o virbr0 -p tcp -m state –state NEW -j ACCEPT else #nothing exit fi

chmod 750 /opt/shell/vm_iptables.sh echo ‘/opt/shell/vm_iptables.sh’ >> /etc/rc.local

9.5让虚传机随宿主启动 cd /etc/xen/auto ln -s ../vm1 ./vm1

9.6重启宿主并测试

四、安装提示 注意 1.安装源建议使用本地的http或nfs服务,并打开相应端口 2.virt-install后面带上–bridge=virbr0 3.默认网关为192.168.122.1,虚拟机ip要在一个网段 配置在/usr/share/libvirt/networks/default.xml 4.虚拟机首次重启乱码,稍等一会后会自动关闭 5.虚拟机开启http服务后注意打相应iptables端口 6.宿主机增加iptables并放在rc.local中

iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 80 -j DNAT –to 192.168.122.11:80 iptables -I FORWARD -i eth0 -o virbr0 -p tcp -m state –state NEW -j ACCEPT

提示 1.开启虚拟机并进入 xm create vm1 -c 2.立即终止一个虚传机 xm destroy vm1 3.进入虚拟机 xm console vm1 4.跳出虚拟机 ctrl+] 5.重排虚拟机id /etc/init.d/xend restart 6.完整删除虚拟机 这个没有找到 7.移除xen yum groupremove Virtualization

参考: http://caishu.name/content/%E4%BD%BF%E7%94%A8xm%E5%9C%A8centos%E4%B8%8A%E5%AE%89%E8%A3%85xen%E7%9A%84%E8%99%9A%E6%8B%9F%E6%9C%BA http://wiki.kartbuilding.net/index.php/Xen_Networking http://cooker.techsnail.com/index.php/XEN,_KVM,_Libvirt_and_IPTables#The_default_iptables_rules_on_a_XEN_physical_host

Posted in 技术, 虚拟化/xen.

Tagged with , .

rev="post-1202" No comments

By C1G – 2011/03/31