给PHP5.2.*打上Hash冲突漏洞补丁

PHP5.2.通过构造Hash冲突可以实现拒绝服务攻击,针对此漏洞官方发布了PHP 5.3.9但不会为此发布PHP 5.2.18. 5.2. 可以打上下面的patch来解决此问题.

https://github.com/laruence/laruence.github.com/tree/master/php-5.2-max-input-vars

目前已知的受影响的语言以及版本有::

Java, 所有版本

JRuby <= 1.6.5

PHP <= 5.3.8, <= 5.4.0RC3

Python, 所有版本

Rubinius, 所有版本

Ruby <= 1.8.7-p356

Apache Geronimo, 所有版本

Apache Tomcat <= 5.5.34, <= 6.0.34, <= 7.0.22

Oracle Glassfish <= 3.1.1

Jetty, 所有版本

Plone, 所有版本

Rack, 所有版本

V8 JavaScript Engine, 所有版本

不受此影响的语言或者修复版本的语言有::

PHP >= 5.3.9, >= 5.4.0RC4

JRuby >= 1.6.5.1

Ruby >= 1.8.7-p357, 1.9.x

Apache Tomcat >= 5.5.35, >= 6.0.35, >= 7.0.23

Oracle Glassfish, N/A (Oracle reports that the issue is fixed in the main codeline and scheduled for a future CPU)

将php从5.2.14升级到5.2.17并打上补丁下载patch https://github.com/laruence/laruence.github.com/zipball/master

到之前的php编译目录

cd src/lempelf/package/ wget http://www.php.net/get/php-5.2.17.tar.gz/from/kr.php.net/mirror wget http://php-fpm.org/downloads/php-5.2.17-fpm-0.5.14.diff.gz tar zxvf php-5.2.17.tar.gz gzip -cd php-5.2.17-fpm-0.5.14.diff.gz |patch -d php-5.2.17 -p1 patching file configure Hunk #7 succeeded at 110645 (offset 1324 lines). Hunk #9 succeeded at 119634 (offset 1324 lines). patching file configure.in patching file libevent/ChangeLog patching file libevent/Makefile.am patching file libevent/Makefile.in patching file libevent/README patching file libevent/aclocal.m4 patching file libevent/autogen.sh patching file libevent/buffer.c patching file libevent/compat/sys/_time.h patching file libevent/compat/sys/queue.h patching file libevent/config.h.in patching file libevent/configure patching file libevent/configure.in patching file libevent/depcomp patching file libevent/devpoll.c patching file libevent/epoll.c patching file libevent/epoll_sub.c patching file libevent/evbuffer.c patching file libevent/event-config.h patching file libevent/event-fpm.h patching file libevent/event-internal.h patching file libevent/event.3 patching file libevent/event.c patching file libevent/event.h patching file libevent/evhttp.h patching file libevent/evport.c patching file libevent/evsignal.h patching file libevent/evutil.c patching file libevent/evutil.h patching file libevent/http-internal.h patching file libevent/http.c patching file libevent/install-sh patching file libevent/kqueue.c patching file libevent/log.c patching file libevent/log.h patching file libevent/min_heap.h patching file libevent/missing patching file libevent/poll.c patching file libevent/select.c patching file libevent/signal.c patching file libevent/strlcpy-internal.h patching file libevent/strlcpy.c patching file main/php_config.h.in patching file sapi/cgi/Makefile.frag patching file sapi/cgi/cgi_main.c patching file sapi/cgi/config9.m4 patching file sapi/cgi/fastcgi.c patching file sapi/cgi/fastcgi.h patching file sapi/cgi/fpm/Makefile.frag patching file sapi/cgi/fpm/acinclude.m4 patching file sapi/cgi/fpm/conf/php-fpm.conf.in patching file sapi/cgi/fpm/config.m4 patching file sapi/cgi/fpm/fpm.c patching file sapi/cgi/fpm/fpm.h patching file sapi/cgi/fpm/fpm_arrays.h patching file sapi/cgi/fpm/fpm_atomic.h patching file sapi/cgi/fpm/fpm_autoconf.h.in patching file sapi/cgi/fpm/fpm_children.c patching file sapi/cgi/fpm/fpm_children.h patching file sapi/cgi/fpm/fpm_cleanup.c patching file sapi/cgi/fpm/fpm_cleanup.h patching file sapi/cgi/fpm/fpm_clock.c patching file sapi/cgi/fpm/fpm_clock.h patching file sapi/cgi/fpm/fpm_conf.c patching file sapi/cgi/fpm/fpm_conf.h patching file sapi/cgi/fpm/fpm_config.h patching file sapi/cgi/fpm/fpm_env.c patching file sapi/cgi/fpm/fpm_env.h patching file sapi/cgi/fpm/fpm_events.c patching file sapi/cgi/fpm/fpm_events.h patching file sapi/cgi/fpm/fpm_php.c patching file sapi/cgi/fpm/fpm_php.h patching file sapi/cgi/fpm/fpm_php_trace.c patching file sapi/cgi/fpm/fpm_php_trace.h patching file sapi/cgi/fpm/fpm_process_ctl.c patching file sapi/cgi/fpm/fpm_process_ctl.h patching file sapi/cgi/fpm/fpm_request.c patching file sapi/cgi/fpm/fpm_request.h patching file sapi/cgi/fpm/fpm_shm.c patching file sapi/cgi/fpm/fpm_shm.h patching file sapi/cgi/fpm/fpm_shm_slots.c patching file sapi/cgi/fpm/fpm_shm_slots.h patching file sapi/cgi/fpm/fpm_signals.c patching file sapi/cgi/fpm/fpm_signals.h patching file sapi/cgi/fpm/fpm_sockets.c patching file sapi/cgi/fpm/fpm_sockets.h patching file sapi/cgi/fpm/fpm_stdio.c patching file sapi/cgi/fpm/fpm_stdio.h patching file sapi/cgi/fpm/fpm_str.h patching file sapi/cgi/fpm/fpm_trace.c patching file sapi/cgi/fpm/fpm_trace.h patching file sapi/cgi/fpm/fpm_trace_mach.c patching file sapi/cgi/fpm/fpm_trace_pread.c patching file sapi/cgi/fpm/fpm_trace_ptrace.c patching file sapi/cgi/fpm/fpm_unix.c patching file sapi/cgi/fpm/fpm_unix.h patching file sapi/cgi/fpm/fpm_worker_pool.c patching file sapi/cgi/fpm/fpm_worker_pool.h patching file sapi/cgi/fpm/init.d/php-fpm.in patching file sapi/cgi/fpm/xml_config.c patching file sapi/cgi/fpm/xml_config.h patching file sapi/cgi/fpm/zlog.c patching file sapi/cgi/fpm/zlog.h unzip laruence-laruence.github.com-43969a1.zip cd php-5.2.17 patch -p1 patching file configure Hunk #1 succeeded at 2176 (offset 11 lines). patching file configure.in patching file main/main.c patching file main/php_globals.h patching file main/php_variables.c patching file main/php_version.h

打好补丁,重编译一遍php

./configure –prefix=/opt/php-5.2.17p1 –with-config-file-path=/opt/php-5.2.17p1/etc –with-mysql=/opt/mysql –with-mysqli=/opt/mysql/bin/mysql_config –with-iconv-dir=/usr/local –with-freetype-dir –with-jpeg-dir –with-png-dir –with-zlib –with-libxml-dir=/usr –disable-rpath –enable-discard-path –enable-safe-mode –enable-bcmath –enable-shmop –enable-sysvsem –enable-inline-optimization –with-curl –with-curlwrappers –enable-mbregex –enable-fastcgi –enable-fpm –enable-force-cgi-redirect –enable-mbstring –with-mcrypt –with-gd –enable-gd-native-ttf –with-openssl –with-mhash –enable-pcntl –enable-sockets –with-xmlrpc –enable-zip –enable-soap –enable-xml –enable-zend-multibyte –disable-debug –disable-ipv6 make ZEND_EXTRA_LIBS=’-liconv’ make install cd ../memcache-3.0.5 make clean /opt/php-5.2.17p1/bin/phpize ./configure –with-php-config=/opt/php-5.2.17p1/bin/php-config make make install cd ../eaccelerator-0.9.6.1 make clean /opt/php-5.2.17p1/bin/phpize ./configure –enable-eaccelerator=shared –with-php-config=/opt/php-5.2.17p1/bin/php-config make make install cd ../PDO_MYSQL-1.0.2 make clean /opt/php-5.2.17p1/bin/phpize ./configure –with-php-config=/opt/php-5.2.17p1/bin/php-config –with-pdo-mysql=/opt/mysql make make install cd ../imagick-2.2.2/ make clean /opt/php-5.2.17p1/bin/phpize ./configure –with-php-config=/opt/php-5.2.17p1/bin/php-config make make install #32位用下面 cp ../ZendOptimizer-3.3.9-linux-glibc23-i386/data/5_2_x_comp/ZendOptimizer.so /opt/php-5.2.17p1/lib/php/extensions/no-debug-non-zts-20060613/ #64位用下面 cp ../ZendOptimizer-3.3.9-linux-glibc23-x86_64/data/5_2_x_comp/ZendOptimizer.so /opt/php-5.2.17p1/lib/php/extensions/no-debug-non-zts-20060613/ mkdir -p /opt/php-5.2.17p1/eaccelerator_cache chown www:website /opt/php-5.2.17p1/eaccelerator_cache/ chmod 770 /opt/php-5.2.17p1/eaccelerator_cache/ touch /opt/php-5.2.17p1/logs/php_error.log chown www:website /opt/php-5.2.17p1/logs/php_error.log chmod 770 /opt/php-5.2.17p1/logs/php_error.log #升级pear (可选) /opt/php-5.2.17p1/bin/pear upgrade pear /opt/php-5.2.17p1/bin/pear install Benchmark Cache_Lite DB HTTP Mail Mail_Mime Net_SMTP Net_Socket Pager XML_Parser XML_RPC cp -p /opt/php/etc/php.ini /opt/php-5.2.17p1/etc/ cp -p /opt/php/etc/php-fpm.conf /opt/php-5.2.17p1/etc/ chown root:website /opt/php-5.2.17p1/etc/* chmod 660 /opt/php-5.2.17p1/etc/* /opt/php/sbin/php-fpm stop #删掉软连接,切换php rm /opt/php ln -s /opt/php-5.2.17p1/ /opt/php /opt/php/sbin/php-fpm start

注意phpfpm.conf,php.ini中的路径

找不到libmysqlclient.so.16

./conftest: error while loading shared libraries: libmysqlclient.so.16

echo /opt/mysql/lib/mysql >> /etc/ld.so.conf ldconfig -v

eAccelerator出错

[eAccelerator] This build of “eAccelerator” was compiled for PHP version 5.2.14. Rebuild it for your PHP version (5.2.17p1) or download precompiled binaries.

重新编译eAccelerator

参考: http://www.laruence.com/2011/12/29/2412.html

Posted in PHP, 安全, 安全通告.

Tagged with PHP, 补丁.

rev="post-1307" No comments

By C1G – 2012/01/12

mysql5.1.26rc升级至Percona mysql5.5.17

mysql5.5 比mysql5.1有更快的innodb,更好的多核支持,加强的复制功能,这正是我看中的. Percona-Server是MySQL的衍生版，在功能和性能上较官方MySQL 又有着很显著的提升.

Percona版和官方mysql.5.5.17下编出中文支持需用 -DWITH_EXTRA_CHARSETS=all 下面参数无法编出 -DWITH_EXTRA_CHARSETS:STRING=utf8,gbk 官方mysql.5.5.18两个参数都无法编译出中文支持… 5.5起使用cmake编译,可以用yum安装 5.6也可用此安装

wget http://www.percona.com/redir/downloads/Percona-Server-5.5/Percona-Server-5.5.17-22.1/source/Percona-Server-5.5.17-rel22.1.tar.gz tar zxvf Percona-Server-5.5.17-rel22.1.tar.gz cd Percona-Server-5.5.17-rel22.1 CC=gcc CFLAGS=”-DBIG_JOINS=1 -DHAVE_DLOPEN=1 -O3″ CXX=g++ CXXFLAGS=”-DBIG_JOINS=1 -DHAVE_DLOPEN=1 -felide-constructors -fno-rtti -O3″ cmake . \ -DCMAKE_BUILD_TYPE:STRING=Release \ -DSYSCONFDIR:PATH=/opt/mysql-5.5.17-22.1 \ -DCMAKE_INSTALL_PREFIX:PATH=/opt/mysql-5.5.17-22.1 \ -DENABLED_PROFILING:BOOL=ON \ -DENABLE_DEBUG_SYNC:BOOL=OFF \ -DMYSQL_DATADIR:PATH=/opt/mysql-5.5.17-22.1/var \ -DMYSQL_MAINTAINER_MODE:BOOL=OFF \ -DWITH_EXTRA_CHARSETS=all \ -DWITH_BIG_TABLES:BOOL=ON \ -DWITH_FAST_MUTEXES:BOOL=ON \ -DENABLE-PROFILING:BOOL=ON \ -DWITH_SSL:STRING=bundled \ -DWITH_UNIT_TESTS:BOOL=OFF \ -DWITH_ZLIB:STRING=bundled \ -DWITH_PARTITION_STORAGE_ENGINE:BOOL=ON \ -DWITH_PLUGINS=heap,csv,partition,innodb_plugin,myisam \ -DEFAULT_COLLATION=utf8_general_ci \ -DEFAULT_CHARSET=utf8 \ -DENABLED_ASSEMBLER:BOOL=ON \ -DENABLED_LOCAL_INFILE:BOOL=ON \ -DENABLED_THREAD_SAFE_CLIENT:BOOL=ON \ -DENABLED_EMBEDDED_SERVER:BOOL=OFF \ -DWITH_CLIENT_LDFLAGS:STRING=all-static \ -DINSTALL_LAYOUT:STRING=STANDALONE \ -DCOMMUNITY_BUILD:BOOL=ON;

失败后更改配制操作

make clean rm -f CMakeCache.txt

编译安装

make make install cp support-files/mysql.server /opt/mysql-5.5.17-22.1/bin/ chown -R mysql:mysql /opt/mysql-5.5.17-22.1 chmod 755 /opt/mysql-5.5.17-22.1/bin/mysql.server ln -s /opt/mysql-5.5.17-22.1/ /opt/mysql

复制5.1全部数据

cp -a /opt/mysql-5.1.26rc/var /opt/mysql-5.5.17-22.1/

复制5.1的原配置文件启动出错

cp /opt/mysql-5.1.26rc/my.cnf /opt/mysql-5.5.17-22.1/ /opt/mysql-5.5.17-22.1/bin/mysqld –skip-grant-tables –user=mysql 111212 15:50:51 [Note] Flashcache bypass: disabled 111212 15:50:51 [Note] Flashcache setup error is : ioctl failed 111212 15:50:51 [Warning] option ‘innodb-autoextend-increment’: unsigned value 52428800 adjusted to 1000 111212 15:50:51 [Note] Plugin ‘InnoDB’ is disabled. 111212 15:50:51 [ERROR] bin/mysqld: unknown option ‘–skip-locking’ 111212 15:50:51 [ERROR] Aborting 111212 15:50:51 [Note] bin/mysqld: Shutdown complete

mysql5.5不再支持skip-locking 参数,需从my.cnf中去除; 如果原先用了innodb还需修改参数和实际文件大小一致

ls -lh /opt/mysql/var/ -rw-r—– 1 mysql mysql 2030043136 Dec 13 16:55 ibdata1 -rw-r—– 1 mysql mysql 67108864 Dec 13 16:55 ib_logfile0 -rw-r—– 1 mysql mysql 67108864 Dec 13 16:37 ib_logfile1

计算ibdata1 文件大小 2030043136/1024/1024=1936M,ib_logfile=67108864/1024/1024=64M

innodb_data_file_path = ibdata1:1936M:autoextend innodb_log_file_size = 64M innodb_log_files_in_group = 2 #后面不需要M innodb-autoextend-increment=50

修改过的my.cnf discuz论坛应用,innodb数据2G左右

[client] port = 3306 socket = /opt/mysql/mysql.sock [mysqld] port = 3306 socket = /opt/mysql/mysql.sock #skip-locking skip-name-resolve back_log=100 key_buffer_size = 4128M query_cache_size = 128M query_cache_limit = 2M #default=1M query_cache_min_res_unit = 2k #default=4K max_allowed_packet = 16M table_cache = 5096 table_definition_cache = 1024 tmp_table_size = 1792M #default=16M max_heap_table_size = 512M #default=16M read_buffer_size = 8M read_rnd_buffer_size = 32M sort_buffer_size = 256M join_buffer_size = 20M myisam_sort_buffer_size = 256M thread_cache_size = 80 #default=0 thread_stack = 192K #default=192K thread_concurrency = 16 #default=10 connect_timeout = 30 max_connection = 500 max_connect_errors = 30 wait_timeout = 30 concurrent_insert=2 #以增加碎片代价提高写入 server-id = 9 innodb_additional_mem_pool_size = 64M innodb_buffer_pool_size = 4G innodb_autoextend_increment =50 #ibdata1以实际文件大小代替 innodb_data_file_path = ibdata1:1936M:autoextend #新增参数 innodb_write_io_threads = 8 #新增参数 innodb_read_io_threads = 8 innodb_thread_concurrency = 16 innodb_flush_log_at_trx_commit = 0 innodb_log_buffer_size = 8M #log_file和原配置保持一样 innodb_log_file_size = 64M innodb_log_files_in_group = 2 innodb_max_dirty_pages_pct = 60 innodb_lock_wait_timeout = 50 #innodb_file_per_table [mysqldump] quick max_allowed_packet = 32M [mysql] no-auto-rehash [isamchk] key_buffer = 256M sort_buffer_size = 256M read_buffer = 8M write_buffer = 8M [myisamchk] key_buffer = 256M sort_buffer_size = 256M read_buffer = 8M write_buffer = 8M [mysqlhotcopy] interactive-timeout

再次跳过权限表启动

/opt/mysql-5.5.17-22.1/bin/mysqld –skip-grant-tables –user=mysql 111213 16:49:29 [Note] Flashcache bypass: disabled 111213 16:49:29 [Note] Flashcache setup error is : ioctl failed 111213 16:49:29 InnoDB: The InnoDB memory heap is disabled 111213 16:49:29 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111213 16:49:29 InnoDB: Compressed tables use zlib 1.2.3 111213 16:49:29 InnoDB: Initializing buffer pool, size = 4.0G 111213 16:49:30 InnoDB: Completed initialization of buffer pool 111213 16:49:30 InnoDB: highest supported file format is Barracuda. InnoDB: 127 rollback segment(s) active. 111213 16:49:30 InnoDB: Waiting for the background threads to start 111213 16:49:31 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 1881683892 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’events_waits_current’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’events_waits_history’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’events_waits_history_long’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’setup_consumers’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’setup_instruments’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’setup_timers’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’performance_timers’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’threads’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’events_waits_summary_by_thread_by_event_name’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’events_waits_summary_by_instance’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’events_waits_summary_global_by_event_name’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’file_summary_by_event_name’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’file_summary_by_instance’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’mutex_instances’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’rwlock_instances’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’cond_instances’ has the wrong structure 111213 16:49:31 [ERROR] Native table ‘performance_schema’.’file_instances’ has the wrong structure 111213 16:49:31 [Note] /opt/mysql-5.5.17-22.1/bin/mysqld: ready for connections. Version: ‘5.5.17’ socket: ‘/opt/mysql/mysql.sock’ port: 3306 Source distribution

升级

/opt/mysql/bin/mysql_upgrade Looking for ‘mysql’ as: /opt/mysql/bin/mysql Looking for ‘mysqlcheck’ as: /opt/mysql/bin/mysqlcheck Running ‘mysqlcheck’ with connection arguments: ‘–port=3306’ ‘–socket=/opt/mysql/mysql.sock’ Running ‘mysqlcheck’ with connection arguments: ‘–port=3306’ ‘–socket=/opt/mysql/mysql.sock’ discuzx.pre_common_addon OK discuzx.pre_common_admincp_cmenu OK discuzx.pre_common_admincp_group OK discuzx.pre_common_admincp_member OK discuzx.pre_common_admincp_perm OK discuzx.pre_common_admincp_session OK discuzx.pre_common_admingroup OK discuzx.pre_common_adminnote OK discuzx.pre_common_advertisement OK discuzx.pre_common_advertisement_custom OK discuzx.pre_common_banned OK discuzx.pre_common_block OK discuzx.pre_common_block_favorite OK discuzx.pre_common_block_item OK discuzx.pre_common_block_item_data OK discuzx.pre_common_block_permission OK discuzx.pre_common_block_pic OK discuzx.pre_common_block_style OK discuzx.pre_common_block_xml OK discuzx.pre_common_cache OK discuzx.pre_common_card OK discuzx.pre_common_card_log OK discuzx.pre_common_card_type OK discuzx.pre_common_credit_log OK discuzx.pre_common_credit_rule OK discuzx.pre_common_credit_rule_log OK discuzx.pre_common_credit_rule_log_field OK discuzx.pre_common_cron OK discuzx.pre_common_district OK discuzx.pre_common_diy_data OK discuzx.pre_common_domain OK discuzx.pre_common_failedlogin OK discuzx.pre_common_friendlink OK discuzx.pre_common_grouppm OK discuzx.pre_common_invite OK discuzx.pre_common_magic OK discuzx.pre_common_magiclog OK discuzx.pre_common_mailcron OK discuzx.pre_common_mailqueue OK discuzx.pre_common_member OK discuzx.pre_common_member_action_log OK discuzx.pre_common_member_connect OK discuzx.pre_common_member_count OK discuzx.pre_common_member_field_forum OK discuzx.pre_common_member_field_home OK discuzx.pre_common_member_grouppm OK discuzx.pre_common_member_log OK discuzx.pre_common_member_magic OK discuzx.pre_common_member_profile OK discuzx.pre_common_member_profile_setting OK discuzx.pre_common_member_security OK discuzx.pre_common_member_stat_field OK discuzx.pre_common_member_stat_fieldcache OK discuzx.pre_common_member_stat_search OK discuzx.pre_common_member_stat_searchcache OK discuzx.pre_common_member_status OK discuzx.pre_common_member_validate OK discuzx.pre_common_member_verify OK discuzx.pre_common_member_verify_info OK discuzx.pre_common_moderate OK discuzx.pre_common_myapp OK discuzx.pre_common_myinvite OK discuzx.pre_common_mytask OK discuzx.pre_common_nav OK discuzx.pre_common_onlinetime OK discuzx.pre_common_onlinetime_bak OK discuzx.pre_common_plugin OK discuzx.pre_common_pluginvar OK discuzx.pre_common_process OK discuzx.pre_common_regip OK discuzx.pre_common_relatedlink OK discuzx.pre_common_report OK discuzx.pre_common_searchindex OK discuzx.pre_common_secquestion OK discuzx.pre_common_session OK discuzx.pre_common_setting OK discuzx.pre_common_smiley OK discuzx.pre_common_sphinxcounter OK discuzx.pre_common_stat OK discuzx.pre_common_statuser OK discuzx.pre_common_style OK discuzx.pre_common_stylevar OK discuzx.pre_common_syscache OK discuzx.pre_common_tag OK discuzx.pre_common_tagitem OK discuzx.pre_common_task OK discuzx.pre_common_taskvar OK discuzx.pre_common_template OK discuzx.pre_common_template_block OK discuzx.pre_common_template_permission OK discuzx.pre_common_uin_black OK discuzx.pre_common_usergroup OK discuzx.pre_common_usergroup_field OK discuzx.pre_common_word OK discuzx.pre_common_word_type OK discuzx.pre_connect_feedlog OK discuzx.pre_connect_memberbindlog OK discuzx.pre_connect_tlog OK discuzx.pre_forum_access OK discuzx.pre_forum_activity OK discuzx.pre_forum_activityapply OK discuzx.pre_forum_announcement OK discuzx.pre_forum_attachment OK discuzx.pre_forum_attachment_0 OK discuzx.pre_forum_attachment_1 OK discuzx.pre_forum_attachment_2 OK discuzx.pre_forum_attachment_3 OK discuzx.pre_forum_attachment_4 OK discuzx.pre_forum_attachment_5 OK discuzx.pre_forum_attachment_6 OK discuzx.pre_forum_attachment_7 OK discuzx.pre_forum_attachment_8 OK discuzx.pre_forum_attachment_9 OK discuzx.pre_forum_attachment_unused OK discuzx.pre_forum_attachtype OK discuzx.pre_forum_bbcode OK discuzx.pre_forum_creditslog OK discuzx.pre_forum_debate OK discuzx.pre_forum_debatepost OK discuzx.pre_forum_faq OK discuzx.pre_forum_forum OK discuzx.pre_forum_forum_threadtable OK discuzx.pre_forum_forumfield OK discuzx.pre_forum_forumrecommend OK discuzx.pre_forum_groupcreditslog OK discuzx.pre_forum_groupfield OK discuzx.pre_forum_groupinvite OK discuzx.pre_forum_grouplevel OK discuzx.pre_forum_groupranking OK discuzx.pre_forum_groupuser OK discuzx.pre_forum_imagetype OK discuzx.pre_forum_medal OK discuzx.pre_forum_medallog OK discuzx.pre_forum_memberrecommend OK discuzx.pre_forum_moderator OK discuzx.pre_forum_modwork OK discuzx.pre_forum_onlinelist OK discuzx.pre_forum_order OK discuzx.pre_forum_poll OK discuzx.pre_forum_polloption OK discuzx.pre_forum_pollvoter OK discuzx.pre_forum_post OK discuzx.pre_forum_post_1 OK discuzx.pre_forum_post_2 OK discuzx.pre_forum_post_tableid OK discuzx.pre_forum_postcomment OK discuzx.pre_forum_postlog OK discuzx.pre_forum_postposition OK discuzx.pre_forum_poststick OK discuzx.pre_forum_promotion OK discuzx.pre_forum_ratelog OK discuzx.pre_forum_relatedthread OK discuzx.pre_forum_replycredit OK discuzx.pre_forum_rsscache OK discuzx.pre_forum_spacecache OK discuzx.pre_forum_statlog OK discuzx.pre_forum_thread OK discuzx.pre_forum_threadclass OK discuzx.pre_forum_threadimage OK discuzx.pre_forum_threadlog OK discuzx.pre_forum_threadmod OK discuzx.pre_forum_threadpartake OK discuzx.pre_forum_threadrush OK discuzx.pre_forum_threadtype OK discuzx.pre_forum_trade OK discuzx.pre_forum_tradecomment OK discuzx.pre_forum_tradelog OK discuzx.pre_forum_typeoption OK discuzx.pre_forum_typeoptionvar OK discuzx.pre_forum_typevar OK discuzx.pre_forum_warning OK discuzx.pre_home_album OK discuzx.pre_home_album_category OK discuzx.pre_home_appcreditlog OK discuzx.pre_home_blacklist OK discuzx.pre_home_blog OK discuzx.pre_home_blog_category OK discuzx.pre_home_blogfield OK discuzx.pre_home_class OK discuzx.pre_home_click OK discuzx.pre_home_clickuser OK discuzx.pre_home_comment OK discuzx.pre_home_docomment OK discuzx.pre_home_doing OK discuzx.pre_home_favorite OK discuzx.pre_home_feed OK discuzx.pre_home_feed_app OK discuzx.pre_home_friend OK discuzx.pre_home_friend_request OK discuzx.pre_home_friendlog OK discuzx.pre_home_notification OK discuzx.pre_home_pic OK discuzx.pre_home_picfield OK discuzx.pre_home_poke OK discuzx.pre_home_pokearchive OK discuzx.pre_home_share OK discuzx.pre_home_show OK discuzx.pre_home_specialuser OK discuzx.pre_home_userapp OK discuzx.pre_home_userappfield OK discuzx.pre_home_viewlog OK discuzx.pre_home_visitor OK discuzx.pre_portal_article_content OK discuzx.pre_portal_article_count OK discuzx.pre_portal_article_related OK discuzx.pre_portal_article_title OK discuzx.pre_portal_article_trash OK discuzx.pre_portal_attachment OK discuzx.pre_portal_category OK discuzx.pre_portal_category_permission OK discuzx.pre_portal_comment OK discuzx.pre_portal_rsscache OK discuzx.pre_portal_topic OK discuzx.pre_portal_topic_pic OK mysql.columns_priv OK mysql.db OK mysql.event OK mysql.func OK mysql.general_log OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.servers OK mysql.slow_log OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK Running ‘mysql_fix_privilege_tables’… OK

再重启下mysql就可以使用. mysql5.1的myisam,innodb共享或单表都可以顺利升到5.5. 需注意字符编码,-skip-locking参数,innodb共享池及日志大小.

Posted in Mysql.

Tagged with innodb, mysql, mysql upgrade, percona.

rev="post-1299" No comments

By C1G – 2011/12/16

mysql InnoDB 版本一览

mysql 5.6.3-m6 的innodb版本为 1.2.3

查看版本方法,可以看到大版本

SELECT * FROM information_schema.plugins;
SELECT @@innodb_version;

MySQL 5.5:

MySQL	Plugin	Status	Date
5.5.18	1.1.8	GA	11/16/2011
5.5.17	1.1.8	GA	10/19/2011
5.5.16	1.1.8	GA	09/15/2011
5.5.15	1.1.8	GA	07/28/2011
5.5.14	1.1.8	GA	07/05/2011
5.5.13	1.1.7	GA	05/31/2011
5.5.12	1.1.6	GA	05/05/2011
5.5.11	1.1.6	GA	04/07/2011
5.5.10	1.1.5	GA	03/15/2011
5.5.9	1.1.5	GA	02/07/2011
5.5.8	1.1.4	GA	12/03/2010
5.5.7	1.1.3	GA	10/14/2010
5.5.6	1.1.2	GA	09/13/2010
5.5.5	1.1.1	Early Adopter	06/06/2010 *³
5.5.4	1.1.0	Beta	04/09/2010 *²
5.5.3	1.0.6	RC	03/24/2010
5.5.2	1.0.6	RC	02/12/2010
5.5.1	1.0.6	RC	01/04/2010
5.5.0	1.0.5	RC	12/07/2009 *¹

*1: The InnoDB Plugin is included in MySQL 5.5 releases as the built-in version of InnoDB.

*2: InnoDB has been upgraded to version 1.1. This version is considered of Beta quality.

*3: InnoDB has been upgraded to version 1.1.1. This version is considered of “early adopter” quality.

MySQL 5.1:

MySQL	Plugin	Status	Date
5.1.60	1.0.17	GA	11/16/2011
5.1.59	1.0.17	GA	09/15/2011
5.1.58	1.0.17	GA	07/05/2011
5.1.57	1.0.16	GA	05/05/2011
5.1.56	1.0.15	GA	03/01/2011
5.1.55	1.0.15	GA	02/07/2011
5.1.54	1.0.14	GA	11/26/2010
5.1.53	1.0.13	GA	11/03/2010
5.1.52	1.0.13	GA	10/11/2010
5.1.51	1.0.12	GA	09/10/2010
5.1.50	1.0.11	GA	08/03/2010
5.1.49	1.0.10	GA	07/09/2010
5.1.48	1.0.9	GA	06/02/2010
5.1.47	1.0.8	GA	05/06/2010
5.1.46	1.0.7	GA	04/06/2010
5.1.45	1.0.6	RC	03/01/2010
5.1.44	1.0.6	RC	02/04/2010
5.1.43	1.0.6	RC	01/15/2010
5.1.42	1.0.6	RC	12/15/2009
5.1.41	1.0.5	RC	11/05/2009 *⁴
5.1.40	1.0.4	Beta	10/06/2009
5.1.39	1.0.4	Beta	09/04/2009
5.1.38	1.0.4	Beta	09/01/2009 *³
x.x.xx	1.0.3	Alpha	03/11/2009
x.x.xx	1.0.2	Alpha	12/01/2008
x.x.xx	1.0.1	Alpha	05/08/2008 *²
x.x.xx	1.0.0	Alpha	04/15/2008 *¹

*1: The initial release of the InnoDB Plugin is based on the built-in InnoDB in MySQL version 5.1.

*2: These 4 versions (1.0.0 – 1.0.3) were only available as separate downloads.

3: 1st Release of Plugin (see 1* below for more details)

*4: InnoDB Plugin has been upgraded to version 1.0.5. This version is considered of Release Candidate (RC) quality.

MariaDB:

MariaDB	Plugin	Status	Date	Comments
5.1.55	1.0.15	GA	03/01/11	If use InnoDB Plugin
5.1.55	1.0.15-12.5	GA	03/01/11	If use built-in InnoDB, which is XtraDB+ in MariaDB *¹
5.2.9	1.0.15-12.7	GA	09/22/11
5.3.2	1.0.15-12.7	GA	10/14/11

*1: XtraDB+ differs some from XtraDB as it contains even further enhancements on top of XtraDB. Please see the following 2 links for more details on these differences:

http://kb.askmonty.org/en/mariadb-53-asynchronous-io-on-windows-with-innodb (work by Wlad) http://www.facebook.com/note.php?note_id=10150211546215933 (work by KNielsen)

There are even some more optimizer additions, but no graphs yet to report.

Changelogs:

Note: The plugin versions had their own changelogs initially, but eventually the changes were just mixed into the general changelogs for the MySQL Server.

Note: Also, some plugin versions span multiple MySQL versions, so those have multiple links below.

Plugin 1.0:

1.0.0  http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-100.html
1.0.1   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-101.html
1.0.2   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-102.html
1.0.3   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-103.html
1.0.4   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-104.html
1.0.5   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-105.html
1.0.6   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-106.html
1.0.7   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-107.html
1.0.8   http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes-108.html
1.0.9   http://dev.mysql.com/doc/refman/5.1/en/news-5-1-48.html
1.0.10  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-49.html
1.0.11  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-50.html
1.0.12  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-51.html
1.0.13  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-52.html
    http://dev.mysql.com/doc/refman/5.1/en/news-5-1-53.html
1.0.14  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-54.html
1.0.15  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-55.html
    http://dev.mysql.com/doc/refman/5.1/en/news-5-1-56.html
1.0.16  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-57.html
1.0.17  http://dev.mysql.com/doc/refman/5.1/en/news-5-1-58.html
    http://dev.mysql.com/doc/refman/5.1/en/news-5-1-59.html
    http://dev.mysql.com/doc/refman/5.1/en/news-5-1-60.html

Plugin 1.1:

1.1.0  http://dev.mysql.com/doc/innodb/1.1/en/innodb-changes-11.html
1.1.1   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-5.html
1.1.2   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-6.html
1.1.3   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-7.html
1.1.4   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-8.html
1.1.5   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-9.html
    http://dev.mysql.com/doc/refman/5.5/en/news-5-5-10.html
1.1.6   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-11.html
    http://dev.mysql.com/doc/refman/5.5/en/news-5-5-12.html
1.1.7   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-13.html
1.1.8   http://dev.mysql.com/doc/refman/5.5/en/news-5-5-14.html
    http://dev.mysql.com/doc/refman/5.5/en/news-5-5-15.html
    http://dev.mysql.com/doc/refman/5.5/en/news-5-5-16.html
    http://dev.mysql.com/doc/refman/5.5/en/news-5-5-17.html
    http://dev.mysql.com/doc/refman/5.5/en/news-5-5-18.html

Misc. Notes:

5.1.38 is first release of the plugin

http://dev.mysql.com/doc/refman/5.1/en/news-5-1-38.html

And for reference, here is the text from that announcement regarding the InnoDB Plugin:

C.1.1. Changes in MySQL 5.1.38

  As of MySQL 5.1.38, the InnoDB Plugin is included in MySQL
  releases, in addition to the built-in version of InnoDB that
  has been included in previous releases. This version of the
  InnoDB Plugin is 1.0.4 and is considered of Beta quality.

  The InnoDB Plugin offers new features, improved performance
  and scalability, enhanced reliability and new capabilities
  for flexibility and ease of use. Among the features of the
  InnoDB Plugin are "Fast index creation," table and index
  compression, file format management, new INFORMATION_SCHEMA
  tables, capacity tuning, multiple background I/O threads, and
  group commit.

  For information about these features, see the InnoDB Plugin
  Manual at

http://www.innodb.com/products/innodb_plugin/plugin-documentation

  For general information about using InnoDB in MySQL,
  see Section 13.6, "The InnoDB Storage Engine."

  The InnoDB Plugin is included in source and binary
  distributions, except RHEL3, RHEL4, SuSE 9 (x86, x86_64,
  ia64), and generic Linux RPM packages.

  To use the InnoDB Plugin, you must disable the built-in
  version of InnoDB that is also included and instruct the
  server to use InnoDB Plugin instead. To accomplish this, use
  the following lines in your my.cnf file:

    [mysqld]
    ignore-builtin-innodb
    plugin-load=innodb=ha_innodb_plugin.so

  For the plugin-load option, innodb is the name to associate
  with the plugin and ha_innodb_plugin.so is the name of the
  shared object library that contains the plugin code. The
  extension of .so applies for Unix (and similar) systems. For
  HP-UX on HPPA (11.11) or Windows, the extension should be .sl
  or .dll, respectively, rather than .so.

  If the server has problems finding the plugin when it starts
  up, specify the pathname to the plugin directory. For
  example, if plugins are located in the lib/mysql/plugin
  directory under the MySQL installation directory and you have
  installed MySQL at /usr/local/mysql, use these lines in your
  my.cnf file:

    [mysqld]
    ignore-builtin-innodb
    plugin-load=innodb=ha_innodb_plugin.so
    plugin_dir=/usr/local/mysql/lib/mysql/plugin

  The previous examples show how to activate the storage engine
  part of InnoDB Plugin, but the plugin also implements several
  InnoDB-related INFORMATION_SCHEMA tables. (For information
  about these tables, see

http://www.innodb.com/doc/innodb_plugin-1.0/innodb-information-schema.html)

  To enable these tables, include additional name=library
  pairs to the plugin-load option:

    [mysqld]
    ignore-builtin-innodb
    plugin-load=innodb=ha_innodb_plugin.so
      ;innodb_trx=ha_innodb_plugin.so
      ;innodb_locks=ha_innodb_plugin.so
      ;innodb_cmp=ha_innodb_plugin.so
      ;innodb_cmp_reset=ha_innodb_plugin.so
      ;innodb_cmpmem=ha_innodb_plugin.so
      ;innodb_cmpmem_reset=ha_innodb_plugin.so

  The plugin-load option here is formatted on multiple lines
  for display purposes but should be written in my.cnf using a
  single line without spaces in the option value. On Windows,
  substitute .dll for each instance of the .so extension.

  After the server starts up, verify that InnoDB Plugin has
  been loaded by using the SHOW PLUGINS statement. For example,
  if you have loaded the storage engine and the
  INFORMATION_SCHEMA tables, the output should include lines
  similar to these:

    mysql> SHOW PLUGINS;
    +---------------------+--------+--------------------+---------------------...
    | Name                | Status | Type               | Library             ...
    +---------------------+--------+--------------------+---------------------...
    | InnoDB              | ACTIVE | STORAGE ENGINE     | ha_innodb_plugin.so ...
    | INNODB_TRX          | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_LOCKS        | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMP          | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMP_RESET    | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMPMEM       | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    | INNODB_CMPMEM_RESET | ACTIVE | INFORMATION SCHEMA | ha_innodb_plugin.so ...
    +---------------------+--------+--------------------+---------------------...

  If you build MySQL from a source distribution, InnoDB Plugin
  is one of the storage engines that is built by default. Build
  MySQL the way you normally do; for example, by using the
  instructions at Section 2.10, "MySQL Installation Using a
  Source Distribution." After the build completes, you should
  find the plugin shared object file under the
  storage/innodb_plugin directory, and make install should
  install it in the plugin directory. Configure MySQL to use
  InnoDB Plugin as described earlier for binary distributions.

Misc Links:

http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-plugin-introduction.html http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-changes.html http://dev.mysql.com/doc/refman/5.5/en/innodb-5-5.html http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-plugin-installation.html

转自:http://www.chriscalender.com/?p=479

Posted in Mysql.

Tagged with innodb, mysql.

rev="post-1296" 1 comment

By C1G – 2011/12/09

内存问题服务器死机一例

硬件:R410 E56062 4G6 Hynix 4GB18-H9 系统:centos5.5

服务器半天左右就会死机一次 tail /var/log/messages

Nov 25 09:28:20 c1g kernel: Machine check events logged Nov 25 09:33:20 c1g kernel: Machine check events logged Nov 25 09:38:20 c1g kernel: Machine check events logged Nov 25 09:43:20 c1g kernel: Machine check events logged Nov 25 09:48:20 c1g kernel: Machine check events logged Nov 25 09:53:20 c1g kernel: Machine check events logged Nov 25 10:03:20 c1g kernel: Machine check events logged Nov 25 10:08:20 c1g kernel: Machine check events logged Nov 25 10:13:20 c1g kernel: Machine check events logged Nov 25 10:18:20 c1g kernel: Machine check events logged Nov 25 10:23:20 c1g kernel: Machine check events logged Nov 25 10:28:20 c1g kernel: Machine check events logged Nov 25 10:44:46 c1g syslogd 1.4.1: restart.

tail -n100 /var/log/mcelog

HARDWARE ERROR. This is *NOT* a software problem! Please contact your hardware vendor CPU 2 BANK 8 TSC 69cd4ba150c [at 2128 Mhz 0 days 0:56:56 uptime (unreliable)] MISC c1ac44000081282 ADDR 5fa5c8580 MCG status: MCi status: Error overflow MCi_MISC register valid MCi_ADDR register valid MCA: MEMORY CONTROLLER RD_CHANNELunspecified_ERR Transaction: Memory read error STATUS cc0001800001009f MCGSTATUS 0 MCE 2 HARDWARE ERROR. This is *NOT* a software problem! Please contact your hardware vendor CPU 6 BANK 8 TSC 69cd4ba18ca [at 2128 Mhz 0 days 0:56:56 uptime (unreliable)] MISC c1ac44000081282 ADDR 5fa5c8580 MCG status: MCi status: Error overflow MCi_MISC register valid MCi_ADDR register valid MCA: MEMORY CONTROLLER RD_CHANNELunspecified_ERR Transaction: Memory read error STATUS cc0001800001009f MCGSTATUS 0 MCE 3 HARDWARE ERROR. This is *NOT* a software problem! Please contact your hardware vendor CPU 4 BANK 8 TSC 69cd4ba1595 [at 2128 Mhz 0 days 0:56:56 uptime (unreliable)] MISC c1ac44000081282 ADDR 5fa5c8580 MCG status: MCi status: Error overflow MCi_MISC register valid MCi_ADDR register valid MCA: MEMORY CONTROLLER RD_CHANNELunspecified_ERR Transaction: Memory read error STATUS cc0001800001009f MCGSTATUS 0

日志中记录了内存出错,原来有24G内存拔掉了一根4G后问题没有再出现.

Posted in 技术.

Tagged with 内存, 死机.

rev="post-1291" No comments

By C1G – 2011/11/30

Lempelf一键包更新到1.0.1

做了点小小更新

ChangeLog

2011-11-30 发布Lempelf 1.0.1 支持32位ZendOptimizer 增加操作用户输入密码提示增加可以自定义ssh端口增加centos6的yum支持增加内核shmmax优化修正eaccelerator.shm_size错误修正php的cgi.fix_pathinfo参数修改net.ipv4.tcp_max_tw_buckets = 15000 nginx更新到0.8.55 不关闭messagebus服务 yum 增加perl-ExtUtils-MakeMaker 安装mysql时自动删除 /etc/my.cnf toolkits下增加采集系统信息脚本

https://blog.c1gstudio.com/lempelfpage

Posted in Lempelf一键包, 技术.

Tagged with Lempelf一键包.

rev="post-1288" No comments

By C1G – 2011/11/30

用varnish来加速图片服务器

varnish是和squid类似的高性能开源HTTP加速器,我这里用来缓存图片,js,css等小文件

varnish cache 192.168.0.15 centos6.0 nagios www后端 192.168.0.11 centos5.3

1.安装varnish

wget http://repo.varnish-cache.org/source/varnish-3.0.0.tar.gz tar zxvf varnish-3.0.0.tar.gz cd varnish-3.0.0 ./configure –prefix=/opt/varnish-3.0.0 make make install ln -s /opt/varnish-3.0.0 /opt/varnish

2.设置权限

cd /opt/varnish #varnish以www:website来运行 chown -R www:website /opt/varnish/var/varnish/ mkdir /var/log/varnish chown -R www:website /var/log/varnish chown -R www:website /opt/varnish/var/varnish/`hostname` mkdir /opt/varnish/var/varnish/`hostname`

3.配置文件

#查看默认配置文件 cat etc/varnish/default.vcl

#编辑新配置文件 vi etc/vcl.conf

#http请求处理过程 #1,receive请求入口状态，根据vcl判断pass还是lookup本地查询 #lookup，在hash表中查找数据，若找到则进入hit状态，否则进入fetch状态 #pass，选择后台，进入fetch状态 #fetch，对请求进行后端的获取，发送请求，获得数据，并进行本地存储 #deliver，将数据发送给客户端，进入done #done,处理结束 backend wwwserver { .host = “192.168.0.11”; .port = “80”; } backend staticserver { .host = “192.168.0.11”; .port = “80”; } acl purge { “localhost”; “127.0.0.1”; “192.168.1.0”/24; } sub vcl_recv { if (req.request == “PURGE”) { if (!client.ip ~ purge) { error 405 “Not allowed.”; } return(lookup); } #去除cookie if (req.request == “GET” && req.url ~ “^/[^?]+\.(jpeg|jpg|png|gif|ico|swf|js|css|txt|zip|html|htm)(\?.*|)$”) { unset req.http.Cookie; } #判断req.http.X-Forwarded-For 如果前端有多重反向代理，这样可以获取客户端IP地址。 if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + “, “+ client.ip; } else { set req.http.X-Forwarded-For = client.ip; } #浏览器Accept-Encoding兼容 if (req.http.Accept-Encoding) { if (req.url ~ “\.(jpg|png|gif|jpeg)$”) { remove req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ “gzip”) { set req.http.Accept-Encoding = “gzip”; } elsif (req.http.Accept-Encoding ~ “deflate”) { set req.http.Accept-Encoding = “deflate”; } else { remove req.http.Accept-Encoding; } } if (req.http.host ~ “^blog.c1gstudio.com”) { set req.backend = wwwserver; if (req.request != “GET” && req.request != “HEAD”) { return(pipe); } elseif(req.url ~ “\.(php|cgi)($|\?)”) { return(pass); } else { return(lookup); } } elsif (req.http.host ~ “^static.c1gstudio.net”) { #第二个域名 set req.backend = staticserver; }else { error 404 “Cache Server”; return(lookup); } } sub vcl_hit { if (req.request == “PURGE”) { set obj.ttl = 0s; error 200 “Purged.”; } } sub vcl_miss { if (req.request == “PURGE”) { error 404 “Not in cache.”; } } sub vcl_fetch { if (req.request == “GET” && req.url ~ “\.(jpeg|jpg|png|gif|ico|swf|js|css|txt|zip)$”) { set beresp.ttl = 600s; unset beresp.http.set-cookie; } else { set beresp.ttl = 3600s; } } #显示是否命中 sub vcl_deliver{ if (obj.hits > 0) { set resp.http.X-Cache = “Server-1-HIT”; set resp.http.X-Cache-Hits = obj.hits; } else { set resp.http.X-Cache = “Server-1-MISS”; } unset resp.http.X-Varnish; set resp.http.Via = “1.1 Xcache”; }

4.启动varnish

/opt/varnish/sbin/varnishd -n /opt/varnish/var/varnish -f /opt/varnish/etc/vcl.conf -a 0.0.0.0:80 -s malloc,1G -g website -u www -T 127.0.0.1:3200 -p sess_workspace=64768 -p thread_pools=2 -p listen_depth=4096 -p first_byte_timeout=10 -p sess_timeout=15 -w 200,5000,10

#参数说明

-n vcache / #临时文件实例名.如果以”/”开头,就必须是一个可用的路径. -a :80 / #服务所在端口.”:80″是默认所有网络都建立80端口,”:”前面是服务器IP. -T :5000 / #管理端口. -s file,/data1/vcache,80g / #虚拟内存文件映射类型,路径以及容量. 包括两种类型”malloc”和”file” -s file,/data2/vcache,80g / #malloc是内存+swap交换模式.很简单.没得说. -s file,/data3/vcache,80g / #file是mmap的文件内存映射机制.(具体情况,参阅”mmap”函数说明) -s file,/data4/vcache,80g / -f /usr/local/varnish/etc/varnish.vcl / #VCL文件路径. -P /var/run/varnish.pid / #PID文件地址. -w 200,5000,10 / #工作进程数.三个参数分别是:,, -h classic,16383 / #hash列表类型,以及长度.默认长度是16383.具体用处和调整实际效果要等我看完源代码才知道. -p user=www / #”-p”是变量配置参数 -p group=website/ #服务运行用户和用户组配置. -p thread_pools=4 / #进程connections pools的个数,数量越多,越耗用cpu和mem,但是处理并发能力越强. #系统手册上说,一个cpu用一个. -p listen_depth=4096 / #TCP队列长度.默认是1024. -p first_byte_timeout=10 #从后端接受第一个字节的超时时间。默认60秒 -p between_bytes_timeout=60 #从后端接收数据后,连接空闲时间,默认60秒 -p sess_timeout=15 #客户端和varnish连接超时时间,默认5秒

===============2015-1-16更新===============

4.2不关闭varnish,重新载入配置

telnet 127.0.0.1 3200 使用命令 vcl.load new.vcl /opt/varnish/etc/vcl.conf (编译出错的话会有提示，成功会返回200) 200 然后使用 vcl.use new.vcl (成功后同样会返回200) 200

crtl+] quit

5.记录日志 /opt/varnish/bin/varnishncsa -n /opt/varnish/var/varnish -w /var/log/varnish/varnish.log &

#定时切割日志 vi /opt/shell/cutvarnishlog.sh

#!/bin/sh # 0 0 * * * /bin/sh /opt/shell/cutvarnishlog.sh > /dev/null 2>&1 date=$(date -d “yesterday” +”%Y%m%d”) pkill -9 varnishncsa mv /var/log/varnish/varnish.log /var/log/varnish/varnish.${date}.log /opt/varnish/bin/varnishncsa -n /opt/varnish/var/varnish -w /var/log/varnish/varnish.log & mkdir -p /var/log/varnish/old gzip -c /var/log/varnish/varnish.${date}.log > /var/log/varnish/old/varnish.${date}.log.gz rm -f /var/log/varnish/varnish.${date}.log rm -f /var/log/varnish/old/varnish$(date -d “-1 month” +”%Y%m*”).log.gz

crontab -e

0 0 * * * /bin/sh /opt/shell/cutvarnishlog.sh > /dev/null 2>&1

6.查看运行统计 /opt/varnish/bin/varnishstat -n /opt/varnish/var/varnish

1+01:13:37 /opt/varnish/var/varnish Hitrate ratio: 10 100 288 Hitrate avg: 0.9987 0.9981 0.9978 22251295 371.40 245.01 client_conn – Client connections accepted 22250487 371.40 245.00 client_req – Client requests received 22185321 371.40 244.29 cache_hit – Cache hits 62904 0.00 0.69 cache_miss – Cache misses 4615 0.00 0.05 backend_conn – Backend conn. success 22 0.00 0.00 backend_fail – Backend conn. failures 59164 0.00 0.65 backend_reuse – Backend conn. reuses 456 0.00 0.01 backend_toolate – Backend conn. was closed 59622 0.00 0.66 backend_recycle – Backend conn. recycles 47470 0.00 0.52 fetch_length – Fetch with Length 16307 0.00 0.18 fetch_chunked – Fetch chunked 2 0.00 0.00 fetch_close – Fetch wanted close 1873 . . n_sess_mem – N struct sess_mem 1834 . . n_sess – N struct sess 655 . . n_object – N struct object 685 . . n_objectcore – N struct objectcore 784 . . n_objecthead – N struct objecthead 405 . . n_waitinglist – N struct waitinglist 2 . . n_vbc – N struct vbc 31 . . n_wrk – N worker threads 381 0.00 0.00 n_wrk_create – N worker threads created 2584 0.00 0.03 n_wrk_queued – N queued work requests 2 . . n_backend – N backends 62227 . . n_expired – N expired objects 5365503 . . n_lru_moved – N LRU moved objects 1362 0.00 0.01 losthdr – HTTP header overflows 18551363 326.47 204.27 n_objwrite – Objects sent with write 22251295 371.40 245.01 s_sess – Total Sessions 22250487 371.40 245.00 s_req – Total Requests 898 0.00 0.01 s_pass – Total pass 63779 0.00 0.70 s_fetch – Total fetch 7539848276 127352.96 83022.43 s_hdrbytes – Total header bytes 141933911830 2248780.45 1562856.20 s_bodybytes – Total body bytes 22251292 371.40 245.01 sess_closed – Session Closed 1 0.00 0.00 sess_herd – Session herd 998035729 16610.26 10989.53 shm_records – SHM records 89193699 1488.60 982.13 shm_writes – SHM writes 328009 8.99 3.61 shm_cont – SHM MTX contention 385 0.00 0.00 shm_cycles – SHM cycles through buffer 1387 0.00 0.02 sms_nreq – SMS allocator requests

7.管理清除缓存 7.1通过Varnish管理端口进行管理 /opt/varnish/bin/varnishadm -T 127.0.0.1:3200 help

CLI connected to 127.0.0.1:3200 help [command] ping [timestamp] auth response quit banner status start stop vcl.load vcl.inline vcl.use vcl.discard vcl.list vcl.show param.show [-l] [ ] param.set panic.show panic.clear storage.list ban.url ban [&& ]… ban.list

通过Varnish管理端口清除缓存,支持正则表达式,1.0时为url.purge参数： /opt/varnish/bin/varnishadm -T 127.0.0.1:3200 ban.url /shanghai-4.html

例：清除所有缓存： /opt/varnish/bin/varnishadm -T 127.0.0.1:3200 ban.url *$

7.2通过telnet方式清除

telnet 127.0.0.1 3200 Trying 127.0.0.1 … Connected to 127.0.0.1. Escape character is ‘^]’. 200 205 —————————– Varnish Cache CLI 1.0 —————————– Linux,2.6.32-71.el6.i686,i686,-smalloc,-smalloc,-hcritbit Type ‘help’ for command list. Type ‘quit’ to close CLI session. help 200 401 help [command] ping [timestamp] auth response quit banner status start stop vcl.load vcl.inline vcl.use vcl.discard vcl.list vcl.show param.show [-l] [ ] param.set panic.show panic.clear storage.list ban.url ban [&& ]… ban.list #1.0时的方法现在不支持 purge.url /shanghai-4.html 200 0 101 44 Unknown request. Type ‘help’ for more info. #正确方法 ban.url /shanghai-4.html 200 0

7.3通过php等其它web请求清除缓存

8.varnish的nginx前端 测试下来nginx和varnish在同一机器上会产生大量time_wait,单独使用没有问题.

upstream mysvr { server 127.0.0.1:82; } server { listen 80; server_name static.c1gstudio.net; index index.html index.htm index.php; root /opt/lampp/htdocs/web; location ~/\.ht { deny all; } location ~(favicon.ico) { log_not_found off; expires 99d; break; } location ~ .*\.(php|html|htm)?$ { return 403; } location / { valid_referers none blocked *.c1gstudio.com *.c1gstudio.net ; if ($invalid_referer) { rewrite ^/ http://leech.c1gstudio.com/leech.gif; return 412; break; } proxy_pass http://mysvr; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } access_log /var/log/nginx/static.c1gstudio.net.log access; }

9.内核优化 vi /etc/sysctl.conf

net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_keepalive_time = 300 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 1 net.ipv4.ip_local_port_range = 5000 65000

sysctl -p

varnish服务器运行基本没有负载

top – 15:54:34 up 34 days, 23:49, 1 user, load average: 0.00, 0.01, 0.00 Tasks: 125 total, 1 running, 124 sleeping, 0 stopped, 0 zombie Cpu(s): 1.8%us, 1.3%sy, 0.0%ni, 95.0%id, 0.4%wa, 0.0%hi, 1.5%si, 0.0%st Mem: 2070548k total, 2017996k used, 52552k free, 83556k buffers Swap: 2097144k total, 0k used, 2097144k free, 1612756k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 26631 www 20 0 228m 134m 81m S 7.6 6.7 74:46.86 varnishd 6070 www 20 0 31852 25m 1000 S 3.3 1.3 7:28.79 nginx 6071 www 20 0 31076 24m 1000 S 2.0 1.2 7:22.34 nginx 6068 www 20 0 31356 25m 976 S 1.7 1.3 7:21.36 nginx

tcp状态 netstat -n | awk ‘/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}’

LAST_ACK 9 SYN_RECV 5 CLOSE_WAIT 3 ESTABLISHED 2083 FIN_WAIT1 95 FIN_WAIT2 247 TIME_WAIT 14412

2011-11-17更新 2011-11-15更新

参考: Varnish-2.1.2 安装与配置.pdf varnish浅析.pdf varnish文件缓存实现2008-11-22.pdf 三个文件包下载varnishdocs 736k

http://blog.s135.com/post/313/ http://eneplace.com/2011/01/varnish-cookies-querystrings.html https://www.varnish-cache.org/docs/3.0/reference/vcl.html#variables

Posted in Squid/varnish.

Tagged with cache, varnish.

rev="post-1272" No comments

By C1G – 2011/11/02

流氓电信DNS劫持强插广告

论坛 BBS 论坛 – body {margin: 0px;padding: 0px;overflow:hidden;} frames[0].location=window.location;function c(){try{var f=frames[0];var d=f.document;(function(s){})(d.readyState);if(d&&(‘complete’==d.readyState)){document.title=d.title?d.title:”;}else{setTimeout(‘c()’,10);}}catch(ex){try{document.domain=document.domain.replace(/^\w+\./,”);c();}catch(ex){}}};c();

论坛右下角偶尔刷新会出现广告弹窗,整个论坛被包含在一个IFRAME里, 本以为是被挂马,搜了下这个111.175.242.23 IP是湖北省武汉市电信,网上很多论坛也遭殃了.

Posted in 安全.

Tagged with dns, 挂马.

rev="post-1265" No comments

By C1G – 2011/10/18

修改numa和io调度优化mysql性能

一.NUMA设置 单机单实例，建议关闭NUMA，关闭的方法有三种： 1.硬件层，在BIOS中设置关闭； 2.OS内核，启动时设置numa=off； 3.可以用numactl命令将内存分配策略修改为interleave（交叉）

方法3 修改mysql.server 330行加上numactl vi /opt/mysql/bin/mysql.server

/usr/bin/numactl –interleave all $bindir/mysqld_safe –datadir=$datadir –pid-file=$server_pid_file $other_args >/dev/null 2>&1 & wait_for_pid created $!; return_value=$?

numastat 查看内存分配

node0 node1 numa_hit 56506002860 201877592362 numa_miss 9099468163 1450668930 numa_foreign 1450668930 9099468163 interleave_hit 6205106 4793392 local_node 56485823400 201848609519 other_node 9119647623 1479651773

二.IO调度算法 Linux有四种IO调度算法：CFQ，Deadline，Anticipatory和NOOP，CFQ是默认的IO调度算法。完全随机的访问环境下，CFQ与Deadline，NOOP性能差异很小，但是一旦有大的连续IO，CFQ可能会造成小IO的响应延时增加，所以数据库环境建议修改为deadline算法，表现更稳定。 IO调度算法都是基于磁盘设计，所以减少磁头移动是最重要的考虑因素之一，但是使用Flash存储设备之后，不再需要考虑磁头移动的问题，可以使用NOOP算法。NOOP的含义就是NonOperation，意味着不会做任何的IO优化，完全按照请求来FIFO的方式来处理IO。

IO调度,默认cfq echo ‘deadline’ > /sys/block/sdb/queue/scheduler cat /sys/block/sdb/queue/scheduler

noop anticipatory [deadline] cfq

减少预读,默认128 echo ’16’ > /sys/block/sda/queue/read_ahead_kb

增大队列,默认128 echo ‘512’ > /sys/block/sda/queue/nr_requests

尽量不使用交换区,默认60 echo ‘0’ > /proc/sys/vm/swappiness

开机运行 vi /etc/rc.local

echo ‘deadline’ > /sys/block/sdb/queue/scheduler echo ’16’ > /sys/block/sda/queue/read_ahead_kb echo ‘512’ > /sys/block/sda/queue/nr_requests

vi /etc/sysctl.conf

vm.swappiness=0

参考: http://blog.wl0.org/2011/03/how-to-start-mysqld-using-numactl/ http://jcole.us/blog/archives/2010/09/28/mysql-swap-insanity-and-the-numa-architecture/ http://linuxcommand.org/man_pages/numactl8.html http://software.intel.com/zh-cn/blogs/2008/11/24/numaxeon1/ http://www.mysqlops.com/2011/07/01/mysql_multi_using_numactl.html https://wickie.hlrs.de/platforms/index.php/Thread_And_Memory_Pinning http://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html http://www.hellodb.net/2011/07/mysql-linux-hardware-tuning.html http://blog.csdn.net/liuben/article/details/5482167 http://hatemysql.com/2011/07/05/mysql%E6%9C%BA%E5%99%A8%E9%85%8D%E7%BD%AE%E6%A0%87%E5%87%86/

Posted in Mysql.

Tagged with io调度, mysql, numa, 优化.

rev="post-1260" No comments

By C1G – 2011/10/09

discuzX2 读写分离及配置mysql复制

之前配置了fastcgi双机负载均衡discuzx2,现在做读写分离

主服务器ip:192.168.0.17 从服务器ip:192.168.0.23

1.主服务器设置 确保在服务器和从服务器上安装的MySQL版本与6.5节，“不同MySQL版本之间的复制兼容性”所示的表兼容。理想情况，应在主服务器和从服务器上使用最近版本的MySQL。在主服务器上为服务器设置一个连接账户。该账户必须授予REPLICATION SLAVE权限。如果账户仅用于复制(推荐这样做)，则不需要再授予任何其它权限。假定你的从服务器ip为192.168.0.23,想要创建用户名为repl的一个账户，从服务器可以使用该账户从你的域内的任何主机使用密码slavepass来访问主服务器。要创建该账户，可使用GRANT语句：

GRANT REPLICATION SLAVE ON *.* TO ‘repl’@’192.168.0.23’ IDENTIFIED BY ‘slavepass’;

更新权限

flush privileges;

确保主服务器主机上my.cnf文件的[mysqld]部分包括一个log-bin选项。该部分还应有一个server-id=Master_id选项，其中master_id必须为1到232–1之间的一个正整数值。例如：

log-bin=/opt/mysql/var_log/mysql-bin expire_logs_days = 10 server-id=1 binlog-do-db=discuzx log-slave-updates

2.从服务器设置 停止用于从服务器的服务器并在其my.cnf文件中添加下面的行： slave_id值同Master_id值一样，必须为1到232–1之间的一个正整数值。并且，从服务器的ID必须与主服务器的ID不相同。例如：

[mysqld] server-id=2 replicate-do-db=discuzx replicate-ignore-table=discuzx.pre_common_session slave-net-timeout=60 master-connect-retry=10 slave-skip-errors #concurrent_insert=2 low-priority-updates=1 max_write_lock_count=1 master-host=192.168.0.17 master-port=3306 master-user=repl master-password=slavepass

3.在主服务器上先锁表，禁止写入操作

FLUSH TABLES WITH READ LOCK；

备份数据库

cd /opt/mysql/ tar -cvf ./mysql-snapshot.tar ./var/discuzx

4.从服务器解压

scp -P 6022 ./mysql-snapshot.tar [email protected]:. tar -xvf ./mysql-snapshot.tar mv ./var/discuzx /opt/mysql/var/

5.设置同步 进入master上的mysql,查看master状态

SHOW MASTER STATUS; File Position Binlog_Do_DB Binlog_Ignore_DB mysql-bin.000001 6416812 discuzx File列显示日志名，而Position显示偏移量。在该例子中，二进制日志值为mysql-bin.000001，偏移量为6416812。记录该值。以后设置从服务器时需要使用这些值。它们表示复制坐标，从服务器应从该点开始从主服务器上进行新的更新。取得快照并记录日志名和偏移量后，可以在主服务器上重新启用写活动：在从服务器上执行下面的语句，用你的系统的实际值替换选项值： CHANGE MASTER TO MASTER_HOST=’192.168.0.17′, MASTER_USER=’repl’, MASTER_PASSWORD=’slavepass’, MASTER_LOG_FILE=’mysql-bin.000001′, MASTER_LOG_POS=6416812; 启动从服务器线程并查看状态： START SLAVE; show slave status; //Slave_IO_Running 与 Slave_SQL_Running 状态都要为Yes //Seconds_Behind_Master为主服务器同频延迟查看进程 show processlist; //应该有两行state值为： Has read all relay log; waiting for the slave I/O thread to update it Waiting for master to send event 复制从I/O线程状态 下面列出了从服务器的I/O线程的State列的最常见的状态。该状态也出现在Slave_IO_State列，由SHOW SLAVE STATUS显示。这说明你可以只通过该语句仔细浏览所发生的事情。 · Connecting to master 线程正试图连接主服务器。 · Checking master version 建立同主服务器之间的连接后立即临时出现的状态。 · Registering slave on master 建立同主服务器之间的连接后立即临时出现的状态。 · Requesting binlog dump 建立同主服务器之间的连接后立即临时出现的状态。线程向主服务器发送一条请求，索取从请求的二进制日志文件名和位置开始的二进制日志的内容。 · Waiting to reconnect after a failed binlog dump request 如果二进制日志转储请求失败(由于没有连接)，线程进入睡眠状态，然后定期尝试重新连接。可以使用–master-connect-retry选项指定重试之间的间隔。 · Reconnecting after a failed binlog dump request 线程正尝试重新连接主服务器。 · Waiting for master to send event 线程已经连接上主服务器，正等待二进制日志事件到达。如果主服务器正空闲，会持续较长的时间。如果等待持续slave_read_timeout秒，则发生超时。此时，线程认为连接被中断并企图重新连接。 · Queueing master event to the relay log 线程已经读取一个事件，正将它复制到中继日志供SQL线程来处理。 · Waiting to reconnect after a failed master event read 读取时(由于没有连接)出现错误。线程企图重新连接前将睡眠master-connect-retry秒。 · Reconnecting after a failed master event read 线程正尝试重新连接主服务器。当连接重新建立后，状态变为Waiting for master to send event。 · Waiting for the slave SQL thread to free enough relay log space 正使用一个非零relay_log_space_limit值，中继日志已经增长到其组合大小超过该值。I/O线程正等待直到SQL线程处理中继日志内容并删除部分中继日志文件来释放足够的空间。 · Waiting for slave mutex on exit 线程停止时发生的一个很简单的状态。 复制从SQL线程状态 下面列出了从服务器的SQL线程的State列的最常见的状态。 · Reading event from the relay log 线程已经从中继日志读取一个事件，可以对事件进行处理了。 · Has read all relay log; waiting for the slave I/O thread to update it 线程已经处理了中继日志文件中的所有事件，现在正等待I/O线程将新事件写入中继日志。 · Waiting for slave mutex on exit 线程停止时发生的一个很简单的状态。 I/O线程的State列也可以显示语句的文本。这说明线程已经从中继日志读取了一个事件，从中提取了语句，并且正在执行语句。最后将master上的表解锁 UNLOCK TABLES；在主服务器上，SHOW PROCESSLIST的输出看上去应为： SHOW PROCESSLIST\G; //state状态应该为Has sent all binlog to slave; waiting for binlog to be updated 复制主线程状态 下面列出了主服务器的Binlog Dump线程的State列的最常见的状态。如果你没有在主服务器上看见任何Binlog Dump线程，这说明复制没有在运行—即，目前没有连接任何从服务器。 · Sending binlog event to slave 二进制日志由各种事件组成，一个事件通常为一个更新加一些其它信息。线程已经从二进制日志读取了一个事件并且正将它发送到从服务器。 · Finished reading one binlog; switching to next binlog 线程已经读完二进制日志文件并且正打开下一个要发送到从服务器的日志文件。 · Has sent all binlog to slave; waiting for binlog to be updated 线程已经从二进制日志读取所有主要的更新并已经发送到了从服务器。线程现在正空闲，等待由主服务器上新的更新导致的出现在二进制日志中的新事件。 · Waiting to finalize termination 线程停止时发生的一个很简单的状态。 6.从服务器添加discuzx用户 在从服务器增加c1g_formaster用户访问从服务器权限 7.设置discuzx读写分离 vi bbs/config/config_global.php $_config[‘db’][‘common’][‘slave_except_table’] = ‘common_session’; #不读从服务器的common_session表 $_config[‘db’][‘slave’] = array(); $_config[‘db’][‘slave’][‘1’][‘dbhost’] = ‘192.168.0.23’; //mysql 从库的host $_config[‘db’][‘slave’][‘1’][‘dbuser’] = ‘c1g_formaster’; //mysql 从库的数据库用户名 $_config[‘db’][‘slave’][‘1’][‘dbpw’] = ‘password’; //mysql 从库的数据库密码 $_config[‘db’][‘slave’][‘1’][‘dbcharset’] = ‘gbk’; $_config[‘db’][‘slave’][‘1’][‘pconnect’] = ‘0’; $_config[‘db’][‘slave’][‘1’][‘dbname’] = ‘discuzx’; //mysql从库的数据库名 $_config[‘db’][‘slave’][‘1’][‘tablepre’] = ‘pre_’; 8.运行效果 内网同步大概会占三四十兆流量; 可能由于我的从服务器还跑着php,负载比较大; 从库运行半天后在更新时会死锁,反而影响了论坛正常运行,还是切回单机; ==============2011-12-06 更新 清除slave设置 在sql里运行 stop slave; reset slave; #这将清除 master.info,relay-log.info 关闭mysql,移险日志 cd /opt/mysql/var/ mv hostname-relay-bin.000609 hostname-relay-bin.000609.bak mv hostname-relay-bin.index hostname-relay-bin.index.bak 重启mysql ==================== 2011-12-19 更新修改主服务器expire_logs_days位置放到log-bin下面 expire_logs_days = 10 参考: http://www.discuz.net/thread-2348271-1-1.html http://liuyu.blog.51cto.com/183345/d-4 http://dev.mysql.com/doc/refman/5.1/zh/replication.html http://blog.lxneng.com/?cat=7

Posted in Discuz/Uchome/Ucenter, Mysql.

Tagged with discuz, mysql, mysql复制.

rev="post-1251" No comments

By C1G – 2011/09/28

inotify+rsync+nginx+fastcgi双机负载均衡discuzx2

一.概述

服务器 web1 R410 E55202 16G SAS300G2 centos5.5 ip:192.168.1.21 web2(new) R410 E56062 16G SAS300G2 centos5.5 ip:192.168.1.23 db R410 E55042 16G SAS146G2 centos5.3

discuz论坛现有500多万贴子,近2万人在线,400万pv/日, 新增web2服务器分担论坛php处理. 尝试过在web2用nfs mount web1的程序来跑,但是访问速度太慢,无法接受改用inotify+rsync效果不错,原来web1负载8左右,现两台各自3.5

web2安装

web2安装好php后配置php-fpm

192.168.1.23:9002 #监听地址及端口 rsyncuser#后面rsync同步的用户 website#同步用户的用户组 128#目前开128个大概占用物理内存4G左右 192.168.1.21,192.168.1.23 #充许请求的服务器(web1,web2自已)

web2打开iptables端口

iptables -A INPUT -p tcp -m tcp -s 192.168.1.21 –dport 9002 -j ACCEPT /etc/init.d/iptables save

db服务器加权限 给web2加上和web1一样的用户访问权限(discuz,ucenter…)

web1安装设置

web1安装inotify https://github.com/rvoicilas/inotify-tools/wiki/ inotify-tools 3.14 is the latest version, released on the 7th of March 2010.

wget –no-check-certificate http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz tar zxvf inotify-tools-3.14.tar.gz cd inotify-tools-3.14 make make install 完成后，注意查看manpage，man inotify、man inotifywait

查看是否支持inotify 从kernel 2.6.13开始正式并入内核，RHEL5已经支持。看看是否有 /proc/sys/fs/inotify/目录，以确定内核是否支持inotify ll /proc/sys/fs/inotify

total 0 -rw-r–r– 1 root root 0 Sep 14 14:01 max_queued_events -rw-r–r– 1 root root 0 Sep 14 14:01 max_user_instances -rw-r–r– 1 root root 0 Sep 14 14:01 max_user_watches

测试inotify /usr/local/bin/inotifywait -mrq –timefmt ‘%d/%m/%y %H:%M’ –format ‘%T %w%f’ -e modify,delete,create,attrib /opt/lampp/htdocs/bbs

16/09/11 15:59 /opt/lampp/htdocs/bbs/data/cache/forum_threadviews_1.log 16/09/11 15:59 /opt/lampp/htdocs/bbs/data/cache/forum_threadviews_1.log 16/09/11 15:59 /opt/lampp/htdocs/bbs/data/cache/forum_threadviews_1.log ….

rsync同步 方法一:使用ssh用户免登录方法二:使用rsync认证免登录

我这里使用方法一

web2增加同步用户 useradd -g website rsyncuser passwd rsyncuser

web1创建公钥并传到web2 ssh-keygen -t rsa

Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is:

复制到web2

scp -P 22 ~/.ssh/id_rsa.pub [email protected]:.

web2设置免登录 将id_rsa.pub导入到.ssh/authorized_keys

cd /home/rsyncuser mkdir .ssh cat id_rsa.pub >> .ssh/authorized_keys chown -R rsyncuser:website .ssh rm id_rsa.pub

可以在web1测试下登录 ssh -p 22 [email protected]

同步脚本 inofity 的exclude支持posix正则,但只能写一个. 注意两边放bbs的路径要一样可以先用rsync同步一下

vi inotify.sh

#!/bin/sh yesterday=`date +%Y%m%d` src=/opt/lampp/htdocs/bbs [email protected]:/opt/lampp/htdocs/ /usr/local/bin/inotifywait -mrq –exclude “data/(threadcache|log|template|sendmail\.lock|cache)” –timefmt ‘%d/%m/%y %H:%M’ –format ‘%T %w%f’ \ -e modify,delete,create,attrib ${src} | while read file do echo -e $(date +%Y-%m-%d_%H:%M:%S)”\r” >>rsynclog/inotify.${yesterday}.log rsync -av –delete –exclude “data/threadcache/” –exclude “data/log/” –exclude “data/template/” –exclude “data/sendmail.lock” ${src} -e “/usr/bin/ssh -p 22” ${des} >>rsynclog/inotify.${yesterday}.log done

在web2上创建不同步的目录和文件 cd /opt/lampp/htdocs/bbs mkdir data/{threadcache,log,template} chown -R rsyncuser:website data/{threadcache,log,template} chmod -R 0775 data/{threadcache,log,template} touch data/sendmail.lock chown -R rsyncuser:website data/sendmail.lock chmod -R 0775 data/sendmail.lock

设置权限并运行 chmod 700 ./inofity.sh ./inofity.sh &

加到开机运行

echo ‘cd /opt/shell && ./inofity.sh & ‘>>/etc/rc.local

cache目录的更新 如果使用memcache可以跳过此步使用文件缓存时 /opt/lampp/htdocs/bbs/data/cache 目录有新注册会员及在线人数等cache,更新很频繁不适于放入inotify中 vi rsync.sh

#!/bin/sh #* * * * * cd /opt/shell && /bin/sh ./rsync.sh > /dev/null 2>&1 yesterday=`date +%Y%m%d` src=/opt/lampp/htdocs/bbs/data/cache [email protected]:/opt/lampp/htdocs/bbs/data/ echo -e $(date +%Y-%m-%d_%H:%M:%S)”\r” >>rsynclog/rsync.${yesterday}.log rsync -av –delete ${src} -e “/usr/bin/ssh -p 22” ${des2} >>rsynclog/rsync.${yesterday}.log

设置权限 chmod 775 ./rsync.sh

crontab -e 加入crontab中 每分钟运行

* * * * * cd /opt/shell && /bin/sh ./rsync.sh > /dev/null 2>&1

第一次同步完后,开启php /opt/php/sbin/php-fpm start

web1开启负载均衡

web1上调整nginx 先创建个test.php来测试,可以在phpinfo里看到不同的机器名或通firebug查看文件header来确定是否在web2上

echo ” > test.php

vi /opt/nginx/conf/nginx.conf

location ~ /test\.php?$ { #fastcgi_pass 127.0.0.1:9000; fastcgi_pass 192.168.1.23:9002; add_header App-Server php2; fastcgi_index index.php; include fcgi.conf; break; }

正式上线只需针对forum.php进行负载均衡,90%的负载都在这文件上(只开启论坛情况下)

upstream backend { ip_hash; #确保分流内网能访问公网,或无对外访问需求 server 192.168.1.23:9002 max_fails=3 fail_timeout=60s; server 127.0.0.1:9000; } location ~ /forum\.php?$ { #fastcgi_pass unix:/tmp/php-cgi.sock; #fastcgi_pass 127.0.0.1:9000; #fastcgi_pass 192.168.1.23:9002; #add_header App-Server php2; fastcgi_pass backend; fastcgi_index index.php; include fcgi.conf; break; }

fcgi.conf

fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx; fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT $document_root; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; # PHP only, required if PHP was built with –enable-force-cgi-redirect fastcgi_param REDIRECT_STATUS 200;

重启nginx reload nginx /bin/kill -HUP cat /dev/shm/nginx.pid

discuz上传附件,后台固定在web1,读贴等操作会依据用户ip分布在两台机器上.

参考: http://www.infoq.com/cn/articles/inotify-linux-file-system-event-monitoring http://hi.baidu.com/tonyty163/blog/item/3c14ca2698672a0a918f9daf.html

Posted in Discuz/Uchome/Ucenter, 网站架构.

Tagged with discuzx, fastcgi, inotify, rsync, 优化, 负载均衡.

rev="post-1248" No comments

By C1G – 2011/09/19

给PHP5.2.*打上Hash冲突漏洞补丁

mysql5.1.26rc升级至Percona mysql5.5.17

mysql InnoDB 版本一览

内存问题服务器死机一例

Lempelf一键包更新到1.0.1

ChangeLog

用varnish来加速图片服务器

crtl+] quit

流氓电信DNS劫持强插广告

修改numa和io调度优化mysql性能

discuzX2 读写分离及配置mysql复制

inotify+rsync+nginx+fastcgi双机负载均衡discuzx2

一.概述

web2安装

web1安装设置

web1开启负载均衡

About C1G军火库

分类

归档

其他操作

近期文章

近期评论

AI

博客互联

安全

我的

技术论坛

收藏夹

架构研发

设计体验

ChangeLog

crtl+] quit

一.概述

web2安装

web1安装设置

web1开启负载均衡

Subscribe

About C1G军火库

分类

归档

其他操作

近期文章

近期评论

标签

AI

博客互联

安全

我的

技术论坛

收藏夹

架构研发

设计体验