Skip to content


开源分布式搜索平台ELK(Elasticsearch+Logstash+Kibana)+Redis+Syslog-ng实现日志实时搜索

logstash + elasticsearch + Kibana+Redis+Syslog-ng

ElasticSearch是一个基于Lucene构建的开源,分布式,RESTful搜索引擎。设计用于云计算中,能够达到实时搜索,稳定,可靠,快速,安装使用方便。支持通过HTTP使用JSON进行数据索引。

logstash是一个应用程序日志、事件的传输、处理、管理和搜索的平台。你可以用它来统一对应用程序日志进行收集管理,提供 Web 接口用于查询和统计。其实logstash是可以被别的替换,比如常见的fluented

Kibana是一个为 Logstash 和 ElasticSearch 提供的日志分析的 Web 接口。可使用它对日志进行高效的搜索、可视化、分析等各种操作。

Redis是一个高性能的内存key-value数据库,非必需安装,可以防止数据丢失.
kibana
参考:
http://www.logstash.net/
http://chenlinux.com/2012/10/21/elasticearch-simple-usage/
http://www.elasticsearch.cn
http://download.oracle.com/otn-pub/java/jdk/7u67-b01/jdk-7u67-linux-x64.tar.gz?AuthParam=1408083909_3bf5b46169faab84d36cf74407132bba
http://curran.blog.51cto.com/2788306/1263416
http://storysky.blog.51cto.com/628458/1158707/
http://zhumeng8337797.blog.163.com/blog/static/10076891420142712316899/
http://enable.blog.51cto.com/747951/1049411
http://chenlinux.com/2014/06/11/nginx-access-log-to-elasticsearch/
http://www.w3c.com.cn/%E5%BC%80%E6%BA%90%E5%88%86%E5%B8%83%E5%BC%8F%E6%90%9C%E7%B4%A2%E5%B9%B3%E5%8F%B0elkelasticsearchlogstashkibana%E5%85%A5%E9%97%A8%E5%AD%A6%E4%B9%A0%E8%B5%84%E6%BA%90%E7%B4%A2%E5%BC%95
http://woodygsd.blogspot.com/2014/06/an-adventure-with-elk-or-how-to-replace.html
http://www.ricardomartins.com.br/enviando-dados-externos-para-a-stack-elk/
http://tinytub.github.io/logstash-install.html

http://jamesmcfadden.co.uk/securing-elasticsearch-with-nginx/
https://github.com/elasticsearch/logstash/blob/master/patterns/grok-patterns
http://zhaoyanblog.com/archives/319.html
http://www.vpsee.com/2014/05/install-and-play-with-elasticsearch/

ip说明
118.x.x.x/16 为客户端ip
192.168.0.39和61.x.x.x为ELK的内网和外网ip

安装JDK

http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

  1. tar zxvf jdk-7u67-linux-x64.tar.gz\?AuthParam\=1408083909_3bf5b46169faab84d36cf74407132b
  2. mv jdk1.7.0_67 /usr/local/
  3. cd /usr/local/
  4. ln -s jdk1.7.0_67 jdk
  5. chown -R root:root jdk/

配置环境变量
vi /etc/profile

  1. export JAVA_HOME=/usr/local/jdk   
  2. export JRE_HOME=$JAVA_HOME/jre
  3. export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:$CLASSPATH
  4. export PATH=$JAVA_HOME/bin:$PATH
  5. export REDIS_HOME=/usr/local/redis
  6. export ES_HOME=/usr/local/elasticsearch
  7. export ES_CLASSPATH=$ES_HOME/config

变量生效
source /etc/profile

验证版本
java -version

  1. java version "1.7.0_67"
  2. Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
  3. Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

如果之前安装过java,可以先卸载
rpm -qa |grep java
java-1.6.0-openjdk-1.6.0.0-1.24.1.10.4.el5
java-1.6.0-openjdk-devel-1.6.0.0-1.24.1.10.4.el5

rpm -e java-1.6.0-openjdk-1.6.0.0-1.24.1.10.4.el5 java-1.6.0-openjdk-devel-1.6.0.0-1.24.1.10.4.el5

安装redis

http://redis.io/

  1. wget http://download.redis.io/releases/redis-2.6.17.tar.gz
  2. tar zxvf redis-2.6.17.tar.gz
  3. mv redis-2.6.17 /usr/local/
  4. cd /usr/local
  5. ln -s redis-2.6.17 redis
  6. cd /usr/local/redis
  7. make
  8. make install

cd utils
./install_server.sh

  1. Please select the redis port for this instance: [6379]
  2. Selecting default: 6379
  3. Please select the redis config file name [/etc/redis/6379.conf]
  4. Selected default - /etc/redis/6379.conf
  5. Please select the redis log file name [/var/log/redis_6379.log]
  6. Selected default - /var/log/redis_6379.log
  7. Please select the data directory for this instance [/var/lib/redis/6379]
  8. Selected default - /var/lib/redis/6379
  9. Please select the redis executable path [/usr/local/bin/redis-server]

编辑配置文件
vi /etc/redis/6379.conf

  1. daemonize yes
  2. port 6379
  3. timeout 300
  4. tcp-keepalive 60

启动
/etc/init.d/redis_6379 start

exists, process is already running or crashed
如报这个错,需要编辑下/etc/init.d/redis_6379,去除头上的\n

加入自动启动
chkconfig –add redis_6379

安装Elasticsearch

http://www.elasticsearch.org/
http://www.elasticsearch.cn
集群安装只要节点在同一网段下,设置一致的cluster.name,启动的Elasticsearch即可相互检测到对方,组成集群

  1. wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.tar.gz
  2. tar zxvf elasticsearch-1.3.2.tar.gz
  3. mv elasticsearch-1.3.2 /usr/local/
  4. cd /usr/local/
  5. ln -s elasticsearch-1.3.2 elasticsearch
  6. elasticsearch/bin/elasticsearch -f
  1. [2014-08-20 13:19:05,710][INFO ][node                     ] [Jackpot] version[1.3.2], pid[19320], build[dee175d/2014-08-13T14:29:30Z]
  2. [2014-08-20 13:19:05,727][INFO ][node                     ] [Jackpot] initializing ...
  3. [2014-08-20 13:19:05,735][INFO ][plugins                  ] [Jackpot] loaded [], sites []
  4. [2014-08-20 13:19:10,722][INFO ][node                     ] [Jackpot] initialized
  5. [2014-08-20 13:19:10,723][INFO ][node                     ] [Jackpot] starting ...
  6. [2014-08-20 13:19:10,934][INFO ][transport                ] [Jackpot] bound_address {inet[/0.0.0.0:9301]}, publish_address {inet[/61.x.x.x:9301]}
  7. [2014-08-20 13:19:10,958][INFO ][discovery                ] [Jackpot] elasticsearch/5hUOX-2ES82s_0zvI9BUdg
  8. [2014-08-20 13:19:14,011][INFO ][cluster.service          ] [Jackpot] new_master [Jackpot][5hUOX-2ES82s_0zvI9BUdg][Impala][inet[/61.x.x.x:9301]], reason: zen-disco-join (elected_as_master)
  9. [2014-08-20 13:19:14,060][INFO ][http                     ] [Jackpot] bound_address {inet[/0.0.0.0:9201]}, publish_address {inet[/61.x.x.x:9201]}
  10. [2014-08-20 13:19:14,061][INFO ][node                     ] [Jackpot] started
  11. [2014-08-20 13:19:14,106][INFO ][gateway                  ] [Jackpot] recovered [0] indices into cluster_state
  12.  
  13.  
  14. [2014-08-20 13:20:58,273][INFO ][node                     ] [Jackpot] stopping ...
  15. [2014-08-20 13:20:58,323][INFO ][node                     ] [Jackpot] stopped
  16. [2014-08-20 13:20:58,323][INFO ][node                     ] [Jackpot] closing ...
  17. [2014-08-20 13:20:58,332][INFO ][node                     ] [Jackpot] closed

ctrl+c退出

以后台方式运行
elasticsearch/bin/elasticsearch -d

访问默认的9200端口
curl -X GET http://localhost:9200

  1. {
  2.   "status" : 200,
  3.   "name" : "Steve Rogers",
  4.   "version" : {
  5.     "number" : "1.3.2",
  6.     "build_hash" : "dee175dbe2f254f3f26992f5d7591939aaefd12f",
  7.     "build_timestamp" : "2014-08-13T14:29:30Z",
  8.     "build_snapshot" : false,
  9.     "lucene_version" : "4.9"
  10.   },
  11.   "tagline" : "You Know, for Search"
  12. }

安装logstash

http://logstash.net/

  1. wget https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz
  2. tar zxvf logstash-1.4.2.tar.gz
  3. mv logstash-1.4.2 /usr/local
  4. cd /usr/local
  5. ln -s logstash-1.4.2 logstash
  6. mkdir logstash/conf
  7. chown -R root:root logstash

logstash

因为java的默认heap size,回收机制等原因,logstash从1.4.0开始不再使用jar运行方式.
以前方式:
java -jar logstash-1.3.3-flatjar.jar agent -f logstash.conf
现在方式:
bin/logstash agent -f logstash.conf

logstash下载即可使用,命令行参数可以参考logstash flags,主要有
http://logstash.net/docs/1.2.1/flags

安装kibana

logstash的最新版已经内置kibana,你也可以单独部署kibana。kibana3是纯粹JavaScript+html的客户端,所以可以部署到任意http服务器上。
http://www.elasticsearch.org/overview/elkdownloads/

  1. wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.0.tar.gz
  2. tar zxvf kibana-3.1.0.tar.gz
  3. mv kibana-3.1.0 /opt/htdocs/www/kibana
  4. vi /opt/htdocs/www/kibana/config.js

配置elasticsearch源
elasticsearch: “http://”+window.location.hostname+”:9200″,

加入iptables
6379为redis端口,9200为elasticsearch端口,118.x.x.x/16为当前测试时的客户端ip

  1. iptables -A INPUT -p tcp -m tcp -s 118.x.x.x/16 --dport 9200 --j ACCEPT

测试运行前端输出
bin/logstash -e ‘input { stdin { } } output { stdout {} }’

输入hello测试
2014-08-20T05:17:02.876+0000 Impala hello

测试运行输出到后端
bin/logstash -e ‘input { stdin { } } output { elasticsearch { host => localhost } }’

访问kibana
http://big.c1gstudio.com/kibana/index.html#/dashboard/file/default.json
Yes- Great! We have a prebuilt dashboard: (Logstash Dashboard). See the note to the right about making it your global default

No results There were no results because no indices were found that match your selected time span

设置kibana读取源
在kibana的右上角有个 configure dashboard,再进入Index Settings
[logstash-]YYYY.MM.DD
这个需和logstash的输出保持一致

elasticsearch 跟 MySQL 中定义资料格式的角色关系对照表如下

MySQL elasticsearch
database index
table type

table schema mapping
row document
field field

ELK整合

syslog-ng.conf

  1. #省略其它内容
  2.  
  3. # Remote logging syslog
  4. source s_remote {
  5.         udp(ip(192.168.0.39) port(514));
  6. };
  7.  
  8. #nginx log
  9. source s_remotetcp {
  10.         tcp(ip(192.168.0.39) port(514) log_fetch_limit(100) log_iw_size(50000) max-connections(50) );
  11. };
  12.  
  13. filter f_filter12     { program('c1gstudio\.com'); };
  14.  
  15. #logstash syslog
  16. destination d_logstash_syslog { udp("localhost" port(10999) localport(10998)  ); };
  17.  
  18. #logstash web
  19. destination d_logstash_web { tcp("localhost" port(10997) localport(10996) ); };
  20.  
  21. log { source(s_remote); destination(d_logstash_syslog); };
  22.  
  23. log { source(s_remotetcp); filter(f_filter12); destination(d_logstash_web); };

logstash_syslog.conf

  1. input {
  2.   udp {
  3.     port => 10999
  4.     type => syslog
  5.   }
  6. }
  7. filter {
  8.   if [type] == "syslog" {
  9.     grok {
  10.       match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
  11.       add_field => [ "received_at", "%{@timestamp}" ]
  12.       add_field => [ "received_from", "%{host}" ]
  13.     }
  14.     syslog_pri { }
  15.     date {
  16.       match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
  17.     }
  18.   }
  19. }
  20.  
  21. output {
  22.   elasticsearch {
  23.   host => localhost   
  24.   index => "syslog-%{+YYYY}"
  25. }
  26. }

logstash_redis.conf

  1. input {
  2.   tcp {
  3.     port => 10997
  4.     type => web
  5.   }
  6. }
  7. filter {
  8.   grok {
  9.     match => [ "message", "%{SYSLOGTIMESTAMP:syslog_timestamp} (?:%{SYSLOGFACILITY:syslog_facility} )?%{SYSLOGHOST:syslog_source} %{PROG:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{IPORHOST:clientip} - (?:%{USER:remote_user}|-) \[%{HTTPDATE:timestamp}\] \"%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:status} (?:%{NUMBER:body_bytes_sent}|-) \"(?:%{URI:http_referer}|-)\" %{QS:agent} (?:%{IPV4:http_x_forwarded_for}|-)"]
  10.     remove_field => [ '@version','host','syslog_timestamp','syslog_facility','syslog_pid']
  11.   }
  12.   date {
  13.     match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  14.   }
  15.    useragent {
  16.         source => "agent"
  17.         prefix => "useragent_"
  18.         remove_field => [ "useragent_device", "useragent_major", "useragent_minor" ,"useragent_patch","useragent_os","useragent_os_major","useragent_os_minor"]
  19.     }
  20.    geoip {
  21.         source => "clientip"
  22.         fields => ["country_name", "region_name", "city_name", "real_region_name", "latitude", "longitude"]
  23.         remove_field => [ "[geoip][longitude]", "[geoip][latitude]","location","region_name" ]
  24.     }
  25. }
  26.  
  27. output {
  28.   #stdout { codec => rubydebug }
  29.  redis {
  30.  batch => true
  31.  batch_events => 500
  32.  batch_timeout => 5
  33.  host => "127.0.0.1"
  34.  data_type => "list"
  35.  key => "logstash:web"
  36.  workers => 2
  37.  }
  38. }

logstash_web.conf

  1. input {
  2.   redis {
  3.     host => "127.0.0.1"
  4.     port => "6379"
  5.     key => "logstash:web"
  6.     data_type => "list"
  7.     codec  => "json"
  8.     type => "web"
  9.   }
  10. }
  11.  
  12. output {
  13.   elasticsearch {
  14.   flush_size => 5000
  15.   host => localhost
  16.   idle_flush_time => 10
  17.   index => "web-%{+YYYY.MM.dd}"
  18.   }
  19.   #stdout { codec => rubydebug }
  20. }

启动elasticsearch和logstash
/usr/local/elasticsearch/bin/elasticsearch -d

/usr/local/logstash/bin/logstash agent -f /usr/local/logstash/conf/logstash_syslog.conf &
/usr/local/logstash/bin/logstash agent -f /usr/local/logstash/conf/logstash_redis.conf &
/usr/local/logstash/bin/logstash agent -f /usr/local/logstash/conf/logstash_web.conf &

关闭
ps aux|egrep ‘search|logstash’
kill pid

安装控制器elasticsearch-servicewrapper
如果是在服务器上就可以使用elasticsearch-servicewrapper这个es插件,它支持通过参数,指定是在后台或前台运行es,并且支持启动,停止,重启es服务(默认es脚本只能通过ctrl+c关闭es)。使用方法是到https://github.com/elasticsearch/elasticsearch-servicewrapper下载service文件夹,放到es的bin目录下。下面是命令集合:
bin/service/elasticsearch +
console 在前台运行es
start 在后台运行es
stop 停止es
install 使es作为服务在服务器启动时自动启动
remove 取消启动时自动启动

vi /usr/local/elasticsearch/service/elasticsearch.conf
set.default.ES_HOME=/usr/local/elasticsearch

命令示例

查看状态
http://61.x.x.x:9200/_status?pretty=true

集群健康查看
http://61.x.x.x:9200/_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign
1409021531 10:52:11 elasticsearch yellow 2 1 20 20 0 0 20

列出集群索引
http://61.x.x.x:9200/_cat/indices?v
health index pri rep docs.count docs.deleted store.size pri.store.size
yellow web-2014.08.25 5 1 5990946 0 3.6gb 3.6gb
yellow kibana-int 5 1 2 0 20.7kb 20.7kb
yellow syslog-2014 5 1 709 0 585.6kb 585.6kb
yellow web-2014.08.26 5 1 1060326 0 712mb 712mb

删除索引
curl -XDELETE ‘http://localhost:9200/kibana-int/’
curl -XDELETE ‘http://localhost:9200/logstash-2014.08.*’

优化索引
$ curl -XPOST ‘http://localhost:9200/old-index-name/_optimize’

查看日志
tail /usr/local/elasticsearch/logs/elasticsearch.log

  1. 2.4mb]->[2.4mb]/[273mb]}{[survivor] [3.6mb]->[34.1mb]/[34.1mb]}{[old] [79.7mb]->[80mb]/[682.6mb]}
  2. [2014-08-26 10:37:14,953][WARN ][monitor.jvm              ] [Red Shift] [gc][young][71044][54078] duration [43s], collections [1]/[46.1s], total [43s]/[26.5m], memory [384.7mb]->[123mb]/[989.8mb], all_pools {[young] [270.5mb]->[1.3mb]/[273mb]}{[survivor] [34.1mb]->[22.3mb]/[34.1mb]}{[old] [80mb]->[99.4mb]/[682.6mb]}
  3. [2014-08-26 10:38:03,619][WARN ][monitor.jvm              ] [Red Shift] [gc][young][71082][54080] duration [6.6s], collections [1]/[9.1s], total [6.6s]/[26.6m], memory [345.4mb]->[142.1mb]/[989.8mb], all_pools {[young] [224.2mb]->[2.8mb]/[273mb]}{[survivor] [21.8mb]->[34.1mb]/[34.1mb]}{[old] [99.4mb]->[105.1mb]/[682.6mb]}
  4. [2014-08-26 10:38:10,109][INFO ][cluster.service          ] [Red Shift] removed {[logstash-Impala-26670-2010][av8JOuEoR_iK7ZO0UaltqQ][Impala][inet[/61.x.x.x:9302]]{client=true, data=false},}, reason: zen-disco-node_failed([logstash-Impala-26670-2010][av8JOuEoR_iK7ZO0UaltqQ][Impala][inet[/61.x.x.x:9302]]{client=true, data=false}), reason transport disconnected (with verified connect)
  5. [2014-08-26 10:39:37,899][WARN ][monitor.jvm              ] [Red Shift] [gc][young][71171][54081] duration [3.4s], collections [1]/[4s], total [3.4s]/[26.6m], memory [411.7mb]->[139.5mb]/[989.8mb], all_pools {[young] [272.4mb]->[1.5mb]/[273mb]}{[survivor] [34.1mb]->[29.1mb]/[34.1mb]}{[old] [105.1mb]->[109mb]/[682.6mb]}

安装bigdesk
要想知道整个插件的列表,请访问http://www.elasticsearch.org/guide/reference/modules/plugins/ 插件还是很多的,个人认为比较值得关注的有以下几个,其他的看你需求,比如你要导入数据当然就得关注river了。

该插件可以查看集群的jvm信息,磁盘IO,索引创建删除信息等,适合查找系统瓶颈,监控集群状态等,可以执行如下命令进行安装,或者访问项目地址:https://github.com/lukas-vlcek/bigdesk

bin/plugin -install lukas-vlcek/bigdesk

  1. Downloading .........................................................................................................................................................................................................................................................DONE
  2. Installed lukas-vlcek/bigdesk into /usr/local/elasticsearch/plugins/bigdesk
  3. Identified as a _site plugin, moving to _site structure ...

cp -ar plugins/bigdesk/_site/ /opt/htdocs/www/bigdesk
访问
http://localhost/bigdesk

安全优化

1.安全漏洞,影响ElasticSearch 1.2及以下版本 http://bouk.co/blog/elasticsearch-rce/
/usr/local/elasticsearch/config/elasticsearch.yml
script.disable_dynamic: true

2.如果有多台机器,可以以每台设置n个shards的方式,根据业务情况,可以考虑取消replias
这里设置默认的5个shards, 复制为0,shards定义后不能修改,replicas可以动态修改
/usr/local/elasticsearch/config/elasticsearch.yml
index.number_of_shards: 5
index.number_of_replicas: 0

#定义数据目录(可选)
path.data: /opt/elasticsearch

3.内存适当调大,初始是-Xms256M, 最大-Xmx1G,-Xss256k,
调大后,最小和最大一样,避免GC, 并根据机器情况,设置内存大小,
vi /usr/local/elasticsearch/bin/elasticsearch.in.sh
if [ “x$ES_MIN_MEM” = “x” ]; then
#ES_MIN_MEM=256m
ES_MIN_MEM=2g
fi
if [ “x$ES_MAX_MEM” = “x” ]; then
#ES_MAX_MEM=1g
ES_MAX_MEM=2g
fi

4.减少shard刷新间隔
curl -XPUT ‘http://61.x.x.x:9200/dw-search/_settings’ -d ‘{
“index” : {
“refresh_interval” : “-1”
}
}’

完成bulk插入后再修改为初始值
curl -XPUT ‘http://61.x.x.x:9200/dw-search/_settings’ -d ‘{
“index” : {
“refresh_interval” : “1s”
}
}’

/etc/elasticsearch/elasticsearch.yml
tranlog数据达到多少条进行平衡,默认为5000,刷新频率,默认为120s
index.translog.flush_threshold_ops: “100000”
index.refresh_interval: 60s

5.关闭文件的更新时间

/etc/fstab

在文件中添加 noatime,nodiratime
/dev/sdc1 /data1 ext4 noatime,nodiratime 0 0

自启动
chkconfig add redis_6379
vi /etc/rc.local
/usr/local/elasticsearch/bin/elasticsearch -d
/usr/local/logstash/bin/logstash agent -f /usr/local/logstash/conf/logstash_syslog.conf &
/usr/local/logstash/bin/logstash agent -f /usr/local/logstash/conf/logstash_redis.conf &
/usr/local/logstash/bin/logstash agent -f /usr/local/logstash/conf/logstash_web.conf &
/opt/lemp startnginx

安装问题

==========================================
LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
See http://jira.codehaus.org/browse/JRUBY-4583

一开始我以为是没有FFI,把jruby,ruby gem都装了一遍.
实际是由于我的/tmp没有运行权限造成的,建个tmp目录就可以了,附上ruby安装步骤.

mkdir /usr/local/jdk/tmp

vi /usr/local/logstash/bin/logstash.lib.sh
JAVA_OPTS=”$JAVA_OPTS -Djava.io.tmpdir=/usr/local/jdk/tmp”

===============================
jruby 安装

  1. wget http://jruby.org.s3.amazonaws.com/downloads/1.7.13/jruby-bin-1.7.13.tar.gz
  2. mv jruby-1.7.13 /usr/local/
  3. cd /usr/local/
  4. ln -s jruby-1.7.13 jruby

Ruby Gem 安装
Ruby 1.9.2版本默认已安装Ruby Gem
安装gem 需要ruby的版本在 1.8.7 以上,默认的centos5 上都是1.8.5 版本,所以首先你的升级你的ruby ,

ruby -v
ruby 1.8.5 (2006-08-25) [x86_64-linux]

  1. wget http://cache.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p547.tar.gz
  2. tar zxvf ruby-1.9.3-p547.tar.gz
  3. cd ruby-1.9.3-p547
  4. ./configure --prefix=/usr/local/ruby-1.9.3-p547
  5. make && make install
  6. cd /usr/local
  7. ln -s ruby-1.9.3-p547 ruby

vi /etc/profile
export PATH=$JAVA_HOME/bin:/usr/local/ruby/bin:$PATH
source /etc/profile

gem install bundler
gem install i18n
gem install ffi

=======================

elasticsearch 端口安全
绑定内网ip

iptables 只开放内网

前端机反向代理
server
{
listen 9201;
server_name big.c1gstudio.com;
index index.html index.htm index.php;
root /opt/htdocs/www;
include manageip.conf;
deny all;

location / {
proxy_pass http://192.168.0.39:9200;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
#proxy_set_header X-Forwarded-For $remote_addr;
add_header X-Cache Cache-156;
proxy_redirect off;
}

access_log /opt/nginx/logs/access.log access;
}

kibana的config.js
elasticsearch: “http://”+window.location.hostname+”:9201″,

Posted in Elasticsearch/Logstash/Kibana, 技术, 日志.

Tagged with , , , , , , .


Linux glibc幽灵漏洞检测及修复方案

幽灵漏洞是Linux glibc库上出现的一个严重的安全问题,他可以让攻击者在不了解系统的任何情况下远程获取操作系统的控制权限。目前他的CVE编号为CVE-2015-0235。

什么是glibc

glibc是GNU发布的libc库,即c运行库。glibc是linux系统中最底层的api,几乎其它任何运行库都会依赖于glibc。glibc除了封装linux操作系统所提供的系统服务外,它本身也提供了许多其它一些必要功能服务的实现。glibc 囊括了几乎所有的 UNIX 通行的标准。

出现了什么漏洞

代码审计公司Qualys的研究人员在glibc库中的__nss_hostname_digits_dots()函数中发现了一个缓冲区溢出的漏洞,这个bug可以经过 gethostbyname*()函数被本地或者远程的触发。应用程序主要使用gethostbyname*()函数发起DNS请求,这个函数会将主机名称转换为ip地址。

漏洞危害

这个漏洞造成了远程代码执行,攻击者可以利用此漏洞获取系统的完全控制权。

漏洞证明

在我们的测试中,我们编写了一个POC,当我们发送一封精心构造的电子邮件给服务器后,我们便可以获得远程Linux服务器的shell,这绕过了目前在32位和64位系统的所有保护(如ASLR,PIE和NX)。

我们能做什么?

给操作系统及时打补丁,我们(Qualys)已与Linux发行商紧密合作,会及时发布补丁。

为什么叫做GHOST?

因为他通过GetHOST函数触发。

哪些版本和操作系统受影响?

第一个受影响的版本是GNU C库的glibc-2.2,2000年11月10号发布。我们已找出多种可以减轻漏洞的方式。我们发现他在2013年5月21号(在glibc-2.17和glibc-2.18发布之间)已经修复。不幸的是他们不认为这是个安全漏洞。从而导致许多稳定版本和长期版本暴露在外,其中包括Debian 7 (wheezy),Red Hat Enterprise,Linux 5 & 6 & 7,CentOS 5 & 6 & 7,Ubuntu 12.04等

修复方案

升级glibc库:

RHEL/CentOS : sudo yum update glibc

Ubuntu : sudo apt-get update ; sudo apt-get install libc6

漏洞测试方法:

wget https://webshare.uchicago.edu/orgs/ITServices/itsec/Downloads/GHOST.c
编译:
gcc -o GHOST GHOST.c

执行:
./GHOST

如果输出:
[root@localhost home]# ./GHOST
not vulnerable

表示漏洞已经修复,如果仅输出“vulnerable”字样,表示漏洞依然存在。

脚本测试漏洞

wget -O GHOST-test.sh http://www.cyberciti.biz/files/scripts/GHOST-test.sh.txt
bash GHOST-test.sh
[root@localhost ~]# bash GHOST-test.sh
Vulnerable glibc version <= 2.17-54 Vulnerable glibc version <= 2.5-122 Vulnerable glibc version <= 2.12-1.148 Detected glibc version 2.12 revision 149 Not Vulnerable. 参考: http://blog.chinaunix.net/uid-509190-id-4807958.html

Posted in 安全通告.

Tagged with .


goaccess分析nginx日志

GoAcces是一款实时日志分析工具.
goaccess_screenshot1M-03L
目前,我们可以通过这款软件查看的统计信息有:

统计概况,流量消耗等
访客排名
动态Web请求
静态web请求,如图片、样式表、脚本等。
来路域名
404 错误
操作系统
浏览器和搜索引擎
主机、DNS和IP地址
HTTP 响应代码
引荐网站
键盘布局
自定义显示
支持超大日志(分析速度很快)

需要用到的几个库文件有:

glib2
GeoIP
ncurses

安装goaccess

  1. yum install glib2 glib2-devel GeoIP-devel ncurses-devel
  2.  
  3. wget http://sourceforge.net/projects/goaccess/files/0.5/goaccess-0.5.tar.gz/download
  4. tar zxvf goaccess-0.5.tar.gz
  5. cd goaccess-0.5
  6. ./configure --enalbe-geoip --enable-utf8
  7. make && make install

GoAccess的基本语法如下:

goaccess [ -b ][ -s ][ -e IP_ADDRESS][ -a ] <-f log_file >
参数说明:

-f – 日志文件名
-b – 开启流量统计,如果希望加快分析速度不建议使用该参数
-s – 开启HTTP响应代码统计
-a – 开启用户代理统计
-e – 开启指定IP地址统计,默认禁用
用法示例:

最简单、常用的命令就是直接调用goaccess命令啦,不带任何影响效率的其他参数

goaccess -f access.log

分析打包文件
zcat access.log.1.gz | goaccess

常见错误:
Your terminal does not support color

vi ~/.bashrc

在最后面添加一行:

export TERM=”xterm-256color”

保存后执行 source ~/.bashrc生效即可

参考:
http://www.linuxde.net/2013/03/12943.html

Posted in 日志.

Tagged with , .


nagios增加监控网卡速率插件check_ethspeed.sh

服务器上线时间长了,网线可能会老化或接触不良导致达不到工作速率.
增加个nagios插件随机监控网卡速率

参阅:linux查看和改变网卡工作速率

cd /opt/nagios/libexec
vi check_ethspeed.sh

  1. #!/bin/bash
  2. #########################################################################
  3. #
  4. # File: check_ethspeed.sh
  5. # Description: Nagios check plugins to check eth speed in *nix.
  6. # Language: GNU Bourne-Again SHell
  7. # Version: 1.0.1
  8. # Date: 2015-1-23
  9. # Author: C1g
  10. # Bog: http://blog.C1gStudio.com
  11. # Note: Allow nagios to run ethtool commands
  12. # visudo
  13. # #Defaults    requiretty
  14. # nagios ALL=NOPASSWD:/sbin/ethtool
  15. #
  16. #########################################################################
  17.  
  18. path=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:~/bin
  19. export PATH
  20. STATE_OK=0
  21. STATE_WARNING=1
  22. STATE_CRITICAL=2
  23. STATE_UNKNOWN=3
  24.  
  25. warn_num=100
  26. critical_num=10
  27. eth=eth0
  28.  
  29. usage(){
  30.   echo -e "Usage: $0 -i|--interface interface -w|--warning warning threshold -c|--critical critical threshold"
  31.   echo -e "Example:"
  32.   echo -e "$0 -i eth0 -w 100 -c 10"
  33.  
  34. }
  35. select_arg(){
  36. if [ $# -eq 0 ];then
  37.   return 1
  38. fi
  39. until [ $# -eq 0 ];do
  40.   case $1 in
  41.     -i|--interface)
  42.       [ $# -lt 2 ] && return 1
  43.       if ! cat /var/log/dmesg |grep $2 >/dev/null 2>&1;then
  44.           return 1
  45.       fi
  46.       eth=$2
  47. shift 2
  48.       ;;
  49.     -w|--warning)
  50.       [ $# -lt 2 ] && return 1
  51.       if ! echo $2 |grep -E -q "^[1-9][0-9]*$";then
  52.         return 1
  53.       fi
  54.       warn_num=$2
  55. shift 2
  56.       ;;
  57.     -c|--critical)
  58.       [ $# -lt 2 ] && return 1
  59.       if ! echo $2 |grep -E -q "^[1-9][0-9]*$";then
  60.         return 1
  61.       fi
  62.       critical_num=$2
  63. shift 2
  64.       ;;
  65.     *)
  66. return 1
  67.      ;;
  68.   esac
  69. done
  70. return 0
  71. }
  72.  
  73. select_arg $@
  74. [ $? -ne 0 ] && usage && exit $STATE_UNKNOWN
  75.  
  76. #echo "warn :$warn_num"
  77. #echo "critical :$critical_num"
  78.  
  79. if [ $critical_num -gt $warn_num ];then
  80.     usage
  81.     exit $STATE_UNKNOWN
  82. fi
  83.  
  84. #ethtool $eth| grep Speed | grep -o  '[0-9]\+'
  85. #kernel >=2.6.33
  86. #cat /sys/class/net/$eth/speed
  87. total=`sudo /sbin/ethtool $eth |grep Speed:|awk '{print $2}' |awk -F 'Mb' '{print $1}'`
  88. if [ $total = Unknown! ];then
  89.     echo "UNKNOWN STATE $eth maybe not working!"
  90.     exit $STATE_UNKNOWN
  91. elif [ $total -gt $warn_num ];then
  92.     echo "$eth OK - Speed: $total Mb/s |$eth=$total;$warn_num;$critical_num;0"
  93.     exit $STATE_OK
  94. elif [ $total -le $warn_num -a $total -gt $critical_num ];then
  95.     echo "$eth WARNING - Speed: $total Mb/s |$eth=$total;$warn_num;$critical_num;0"
  96.     exit $STATE_WARNING
  97. elif [ $total -le $critical_num ];then
  98.     echo "$eth CRITICAL - Speed: $total Mb/s |$eth=$total;$warn_num;$critical_num;0"
  99.     exit $STATE_CRITICAL
  100. else
  101.     echo "UNKNOWN STATE"
  102.     exit $STATE_UNKNOWN
  103. fi

下载check_ethspeed.sh

chown nagios:nagios check_ethspeed.sh
chmod 775 check_ethspeed.sh

运行ethtool命令需root权限
visudo

  1. Defaults    requiretty

注释这一行

添加nagios用户无需密码运行ethtool权限

  1. nagios  ALL=NOPASSWD:/sbin/ethtool

客户端nrpe增加监控命令
echo ‘command[check_ethspeed2]=/opt/nagios/libexec/check_ethspeed.sh -i eth2 -w 100 -c 10’ >> /opt/nagios/etc/nrpe.cfg

重启nrpe
kill `cat /var/run/nrpe.pid`
/opt/nagios/bin/nrpe -c /opt/nagios/etc/nrpe.cfg -d

监控端增加监控服务
vi c1gstudio.cfg

  1. define service{
  2.         use                             local-service,srv-pnp         ; Name of service template to use
  3.         host_name                       c1gstudio
  4.         service_description             check_ethspeed eth2
  5.         check_command                   check_ethspeed!eth2!100!10
  6. notifications_enabled 0
  7.         }

重启nagios
/etc/init.d/nagios reload

check_ethspeed

参阅:http://blog.c1gstudio.com/archives/1748

Posted in Nagios.

Tagged with , , .


给nagios增加监控当前php进程数的插件,并用pnp出图

脚本说明
脚本默认监控为php-fpm 以TCP方式运行在本机的端口php
php-fpm.conf例

  1. <value name="listen_address">127.0.0.1:9000/value>

其它地址需在脚本中修改相应地址127.0.0.1:9000

最终输出的$total processes为当前正在执行或等待的php数,此数为0或越少越好.

vi check_phpprocs.sh

  1. #!/bin/bash
  2. #########################################################################
  3. #
  4. # File:         check_phpprocs.sh
  5. # Description:  Nagios check plugins to check php process in *nix.
  6. # Language:     GNU Bourne-Again SHell
  7. # Version:      1.0.0
  8. # Date:         2015-1-16
  9. # Author:       C1g
  10. # Bog:          http://blog.C1gStudio.com
  11. #########################################################################
  12.  
  13. path=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:~/bin
  14. export PATH
  15. STATE_OK=0
  16. STATE_WARNING=1
  17. STATE_CRITICAL=2
  18. STATE_UNKNOWN=3
  19.  
  20. warn_num=100
  21. critical_num=200
  22.  
  23. usage(){
  24.   echo -e "Usage: $0  -w|--warning warning threshold -c|--critical critical threshold"
  25. }
  26. select_arg(){
  27. if [ $# -eq 0 ];then
  28.   return 1
  29. fi
  30. until [ $# -eq 0 ];do
  31.   case $1 in
  32.     -w|--warning)
  33.       [ $# -lt 2 ] && return 1
  34.       if ! echo $2 |grep -E -q "^[1-9][0-9]*$";then
  35.         return 1
  36.       fi
  37.       warn_num=$2
  38.          shift 2
  39.       ;;
  40.     -c|--critical)
  41.       [ $# -lt 2 ] && return 1
  42.       if ! echo $2 |grep -E -q "^[1-9][0-9]*$";then
  43.         return 1
  44.       fi
  45.       critical_num=$2
  46.         shift 2
  47.       ;;
  48.     *)
  49.         return 1
  50.      ;;
  51.   esac
  52. done
  53. return 0
  54. }
  55.  
  56. select_arg $@
  57. [ $? -ne 0 ] && usage && exit $STATE_UNKNOWN
  58.  
  59. #echo "warn :$warn_num"
  60. #echo "critical :$critical_num"
  61.  
  62. if [ $critical_num -lt $warn_num ];then
  63.     usage
  64.     exit $STATE_UNKNOWN
  65. fi
  66.  
  67. total=`netstat -n | grep 127.0.0.1:9000 | wc -l`
  68. if [ $total -lt $warn_num ];then
  69.     echo "PHP OK - $total processes |PHP=$total;$warn_num;$critical_num;0"
  70.     exit $STATE_OK
  71. elif [ $total -ge $warn_num -a $total -lt $critical_num ];then
  72.     echo "PHP WARNING - $total processes |PHP=$total;$warn_num;$critical_num;0"
  73.     exit $STATE_WARNING
  74. elif [ $total -ge $critical_num ];then
  75.     echo "PHP CRITICAL - $total processes |PHP=$total;$warn_num;$critical_num;0"
  76.     exit $STATE_CRITICAL
  77. else
  78.     echo "UNKNOWN STATE"
  79.     exit $STATE_UNKNOWN
  80. fi

增加执行权限,宫户端nrpe路径为/opt/nagios/
chown nagios:nagios check_phpprocs.sh
chmod 755 check_phpprocs.sh
mv ./check_phpprocs.sh /opt/nagios/libexec/

编辑宫户端nrpe.cfg增加监控命令,这里设置waring阀值为100,critical阀值为200
vi /opt/nagios/etc/nrpe.cfg

  1. command[check_phpprocs]=/opt/nagios/libexec/check_phpprocs.sh -w 100 -c 200

重启宫户端nrpe

  1. kill `cat /var/run/nrpe.pid`
  2. /opt/nagios/bin/nrpe -c /opt/nagios/etc/nrpe.cfg -d

编辑监控端主机文件c1gstudio.cfg,添加监控服务
vi /usr/local/nagios/etc/objects/c1gstudio.cfg

define service{
use local-service,srv-pnp ; Name of service template to use
host_name c1gstudio
service_description PHP Processes
check_command check_nrpe!check_phpprocs
notifications_enabled 1
}
pnp4nagios已配好,srv-pnp为配好的模板 可以直接出图
templates.cfg

  1. define service {
  2.   name       srv-pnp
  3.   register   0
  4.   action_url /pnp/index.php?host=$HOSTNAME$&srv=$SERVICEDESC$
  5.   process_perf_data               1

参见:http://blog.c1gstudio.com/archives/552

重启nagios
/etc/init.d/nagios reload

查看状态
nagios_checkphpprocs

下载check_phpprocs.zip check_phpprocs

参考:http://blog.csdn.net/xluren/article/details/17724043

Posted in Nagios.

Tagged with , , , .


用js创建隐藏来源不带referer的超链接

最直接的是用window.open,对IE9以下有效,Firefox无效

  1. function openwin2(strurl){
  2. window.open(strurl, "newwin", "height=650,width=778,scrollbars=10,resizable=yes");
  3. }

进阶版,IE6会报错

  1. function open_new_window(full_link){
  2.     window.open('javascript:window.name;', '<script>location.replace("'+full_link+'")<\/script>');
  3.  }

高级版,用基于HTML5标准rel=”noreferrer” 并配合noreferrer.js
可以自动识别浏览器并选择最优方案
分为prototype.js




和jquery版




external link

注意将nofrerrer.js中的google地址改成百度的.
http://www.baidu.com/link?url?q

参考:
http://zhongfox.github.io/blog/javascript/2013/08/16/remove-referer-using-js/
https://github.com/knu/noreferrer

Posted in JavaScript/DOM/XML.

Tagged with , .


解决discuzx3.2论坛群发短消息(pm)

一开始以为是程序有漏洞,看了source\include\spacecp\spacecp_pm.php代码才知道有开关可以控制.

  1. 后台->站点功能->其它->
  2.  
  3. 全站是否默认只接受好友短消息:
  4.  是 否
  5. 选择“是”将在个人短消息设置中,默认只接收好友的短消息

选择”是”

  1. 用户->用户组->(选择起始的几个用户组)->基本设置->允许发送短消息:
  2. 是否可以给任何人发短消息:
  3.  是 否
  4. 选择否的话,当对方设置为只接受好友短消息,将无法对其发送短消息

选择”否”

并可以相应结合24小时内发布短消息最大数,并设置发送短消息需消耗积分能设置.

Posted in Discuz/Uchome/Ucenter.

Tagged with , .


使用HAProxy给MySQL slave群进行负载均衡和状态监控

blog_haproxy

一.安装haproxy

haproxy机器
http://haproxy.1wt.deu
需翻墙

  1. tar zxvf haproxy-1.4.25.tar.gz
  2. cd haproxy-1.4.25
  3. make TARGET=linux26
  4. make install
  5. mkdir -p /usr/local/haproxy/
  6. chown nobody:nobody /usr/local/haproxy/
  7. mkdir /etc/haproxy/
  8. cp examples/haproxy.cfg /etc/haproxy/
  9.  
  10. cp examples/haproxy.init /etc/init.d/haproxy
  11. chown root:root /etc/init.d/haproxy
  12. chmod 700 /etc/init.d/haproxy

修改haproxy启动脚本

/usr/sbin/$BASENAME
改成
/usr/local/sbin/$BASENAME

  1. sed -i -r 's|/usr/sbin|/usr/local/sbin|' /etc/init.d/haproxy

编辑配置文件
vi /etc/haproxy/haproxy.cfg

  1. global
  2. #log 127.0.0.1 local0
  3. log 127.0.0.1 local3 info
  4. #log loghost local0 info
  5. maxconn 4096
  6. chroot /usr/local/haproxy
  7. uid nobody
  8. gid nobody
  9. daemon
  10. debug
  11. #quiet
  12.  
  13. defaults
  14. log global
  15. mode tcp
  16. #option httplog
  17. option dontlognull
  18. retries 3
  19. option redispatch
  20. maxconn 2000
  21. contimeout 5000
  22. clitimeout 50000
  23. srvtimeout 50000
  24.  
  25. frontend mysql
  26. bind 192.168.0.107:3306
  27. maxconn 3000
  28. default_backend mysql_slave
  29.  
  30. backend mysql_slave 
  31. #cookie SERVERID rewrite
  32. mode tcp
  33. balance roundrobin
  34. #balance source
  35. #balance leastconn
  36. contimeout 10s
  37. timeout check 2s
  38. option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www
  39. server mysql_192_168_0_104_3306 192.168.0.104:3306 weight 1 check port 9300 inter 5s rise 2 fall 3
  40. server mysql_192_168_0_104_3307 192.168.0.104:3307 weight 1 check port 9301 inter 5s rise 2 fall 3
  41. #server mysql_192_168_0_106_3306 192.168.0.106:3306 weight 1 check port 9300 inter 5s rise 2 fall 3
  42.  
  43. listen  admin_status
  44. mode  http
  45. bind 192.168.0.107:8000
  46. option httplog
  47. log global
  48. stats enable
  49. stats refresh 30s
  50. stats hide-version
  51. stats realm Haproxy\ Statistics
  52. stats uri  /admin-status
  53. stats auth  admin:123456
  54. stats admin if TRUE

打开监控的iptables

  1. iptables -A INPUT -p tcp -m tcp -s 192.168.0.0/24 --dport 8000 -j ACCEPT

添加自启动并启动haproxy服务

  1. chkconfig –add haproxy
  2. chkconfig haproxy on
  3. service haproxy start

被监控机上

我这里是单机双实例,所以有2个脚本,单机只需一个脚本和一个服务端口就行
编辑mysql检测3306脚本
vi /opt/shell/mysqlchk_status_3306.sh

  1. #!/bin/bash
  2. #
  3. # /usr/local/bin/mysqlchk_status.sh
  4. #
  5. # This script checks if a mysql server is healthy running on localhost. It will
  6. # return:
  7. #
  8. # "HTTP/1.x 200 OK\r" (if mysql is running smoothly)
  9. #
  10. # – OR –
  11. #
  12. # "HTTP/1.x 503 Internal Server Error\r" (else)
  13. #
  14.  
  15. MYSQL_HOST="localhost"
  16. MYSQL_PORT="3306"
  17. MYSQL_USERNAME="mysqlcheck"
  18. MYSQL_PASSWORD="paSSword"
  19. MYSQL_PATH="/opt/mysql/bin/"
  20.  
  21. #
  22. # We perform a simple query that should return a few results
  23. #${MYSQL_PATH}mysql -h${MYSQL_HOST} -P${MYSQL_PORT} -u${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -e "show slave status\G;" >/tmp/rep${MYSQL_PORT}.txt
  24. ${MYSQL_PATH}mysql -h${MYSQL_HOST} -P${MYSQL_PORT} -u${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -e "show full processlist;" >/tmp/processlist${MYSQL_PORT}.txt
  25. ${MYSQL_PATH}mysql -h${MYSQL_HOST} -P${MYSQL_PORT} -u${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -e "show slave status\G;" >/tmp/rep${MYSQL_PORT}.txt
  26. iostat=`grep "Slave_IO_Running" /tmp/rep${MYSQL_PORT}.txt  |awk '{print $2}'`           
  27. sqlstat=`grep "Slave_SQL_Running" /tmp/rep${MYSQL_PORT}.txt |awk '{print $2}'`           
  28. result=$(cat /tmp/processlist${MYSQL_PORT}.txt|wc -l)
  29. echo iostat:$iostat and sqlstat:$sqlstat
  30. # if slave_IO_Running and Slave_sql_Running ok,then return 200 code
  31. if [ "$result" -gt "3" ] && [ "$iostat" = "Yes" ] && [ "$sqlstat" = "Yes" ];
  32.  
  33. then
  34.         # mysql is fine, return http 200
  35.         /bin/echo -e "HTTP/1.1 200 OK\r\n"
  36.         
  37. else
  38.         # mysql is down, return http 503
  39.         /bin/echo -e "HTTP/1.1 503 Service Unavailable\r\n"
  40.         
  41. fi

vi /opt/shell/mysqlchk_status_3307.sh

  1. #!/bin/bash
  2. #
  3. # /usr/local/bin/mysqlchk_status.sh
  4. #
  5. # This script checks if a mysql server is healthy running on localhost. It will
  6. # return:
  7. #
  8. # "HTTP/1.x 200 OK\r" (if mysql is running smoothly)
  9. #
  10. # – OR –
  11. #
  12. # "HTTP/1.x 503 Internal Server Error\r" (else)
  13. #
  14.  
  15. MYSQL_HOST="localhost"
  16. MYSQL_PORT="3307"
  17. MYSQL_USERNAME="mysqlcheck"
  18. MYSQL_PASSWORD="paSSword"
  19. MYSQL_PATH="/opt/mysql/bin/"
  20.  
  21. #
  22. # We perform a simple query that should return a few results
  23. #${MYSQL_PATH}mysql -h${MYSQL_HOST} -P${MYSQL_PORT} -u${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -e "show slave status\G;" >/tmp/rep${MYSQL_PORT}.txt
  24. ${MYSQL_PATH}mysql -S/data/mysql/mysql.sock -u${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -e "show full processlist;" >/tmp/processlist${MYSQL_PORT}.txt
  25. ${MYSQL_PATH}mysql -S/data/mysql/mysql.sock -u${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -e "show slave status\G;" >/tmp/rep${MYSQL_PORT}.txt
  26. iostat=`grep "Slave_IO_Running" /tmp/rep${MYSQL_PORT}.txt  |awk '{print $2}'`           
  27. sqlstat=`grep "Slave_SQL_Running" /tmp/rep${MYSQL_PORT}.txt |awk '{print $2}'`           
  28. result=$(cat /tmp/processlist${MYSQL_PORT}.txt|wc -l)
  29. #echo iostat:$iostat and sqlstat:$sqlstat
  30. echo $result
  31. # if slave_IO_Running and Slave_sql_Running ok,then return 200 code
  32. if [ "$result" -gt "3" ] && [ "$iostat" = "Yes" ] && [ "$sqlstat" = "Yes" ];
  33. then
  34.         # mysql is fine, return http 200
  35.         /bin/echo -e "HTTP/1.1 200 OK\r\n"
  36.         
  37. else
  38.         # mysql is down, return http 503
  39.         /bin/echo -e "HTTP/1.1 503 Service Unavailable\r\n"
  40.         
  41. fi

chmod 775 /opt/shell/mysqlchk_status_3306.sh
chmod 775 /opt/shell/mysqlchk_status_3307.sh

在mysql slave另行建立一个具有process和slave_client权限的账号。

  1. CREATE USER 'mysqlcheck'@'localhost' IDENTIFIED BY 'PaSSword';
  2.  
  3. GRANT PROCESS , REPLICATION CLIENT ON * . * TO 'mysqlcheck'@'localhost' IDENTIFIED BY 'PaSSword' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ;
  4.  
  5. flush privileges;

测试脚本
./mysqlchk_status_3306.sh

添加服务
绑定内网ip,运行于930端口,只开放给192.168.0内网
yum install -y xinetd
vim /etc/xinetd.d/mysql_status

  1. service mysqlchk_status3306
  2. {
  3.         flags           = REUSE
  4.         socket_type     = stream
  5.         bind            = 192.168.0.104
  6.         port            = 9300
  7.         wait            = no
  8.         user            = nobody
  9.         server          = /opt/shell/mysqlchk_status_3306.sh
  10.         log_type        = FILE /dev/null
  11.         log_on_failure  += USERID
  12.         disable         = no
  13.         only_from       = 192.168.0.0/24
  14. }
  15. service mysqlchk_status3307
  16. {
  17.         flags           = REUSE
  18.         socket_type     = stream
  19.         bind            = 192.168.0.104
  20.         port            = 9301
  21.         wait            = no
  22.         user            = nobody
  23.         server          = /opt/shell/mysqlchk_status_3307.sh
  24.         log_type        = FILE /dev/null
  25.         log_on_failure  += USERID
  26.         disable         = no
  27.         only_from       = 192.168.0.0/24
  28. }

bind和only_from的ip地址要有haproxy能请求的权限,使用drbd用0.0.0.0
user要用server脚本的执行权限
port端口要在/etc/service 中声明

chattr -i /etc/services
vi /etc/services

  1. mysqlchk_status3306    9300/tcp #haproxy mysql check
  2. mysqlchk_status3307    9301/tcp #haproxy mysql check

services中的mysqlchk_status3306 要和xinetd.d中service名对应

打开iptables

  1. iptables -A INPUT -p tcp -m tcp -s 192.168.0.0/24 --dport 9300 -j ACCEPT
  2. iptables -A INPUT -p tcp -m tcp -s 192.168.0.0/24 --dport 9301 -j ACCEPT

/etc/init.d/iptables save

添加自启动及启动服务
chkconfig xinetd –level 345 on
/etc/init.d/xinetd start

查看是否运行
netstat -lntp

  1. Active Internet connections (only servers)
  2. Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
  3. tcp        0      0 0.0.0.0:9300                0.0.0.0:*                   LISTEN      4863/xinetd         
  4. tcp        0      0 0.0.0.0:9301                0.0.0.0:*                   LISTEN      4863/xinetd

如果没有的话注意检测下bind地址及服务端口

在监控机运行测试
telnet 192.168.0.104 9300

  1. Trying 192.168.0.104...
  2. Connected to 192.168.0.104 (192.168.0.104).
  3. Escape character is '^]'.
  4. /opt/shell/mysqlchk_status_3306.sh: line 24: /tmp/processlist3306.txt: Permission denied
  5. /opt/shell/mysqlchk_status_3306.sh: line 25: /tmp/rep3306.txt: Permission denied
  6. HTTP/1.1 200 OK
  7.  
  8. Connection closed by foreign host.

之前用root运行过所以报错,在被监控机删除临时文件

  1. rm -f /tmp/processlist3306.txt /tmp/processlist3307.txt
  2. rm -f /tmp/rep3306.txt /tmp/rep3307.txt

没有输出则需检查mysqlchk_status_3306.sh脚本执行权限

启动后/var/log/messages 中会有很多日志

  1. Oct 23 14:37:00 lova xinetd[11057]: START: mysqlchk_status3306 pid=11464 from=192.168.0.22
  2. Oct 23 14:37:00 lova xinetd[11057]: EXIT: mysqlchk_status3306 status=0 pid=11464 duration=0(sec)
  3. Oct 23 14:37:05 lova xinetd[11057]: START: mysqlchk_status3306 pid=11494 from=192.168.0.22
  4. Oct 23 14:37:05 lova xinetd[11057]: EXIT: mysqlchk_status3306 status=0 pid=11494 duration=0(sec)

在haproxy配置中将日志输出到黑洞
log_type = FILE /dev/null

查看监控

直接访问localhost是503
http://localhost/
503 Service Unavailable

No server is available to handle this request.

加上admin-status
http://localhost/admin-status

应用时需在slave mysql上的mysql添加通过haproxy的用户权限

haproxy的命令
/etc/init.d/haproxy
Usage: haproxy {start|stop|restart|reload|condrestart|status|check}


优化time_wait,防止端口耗尽
vi /etc/sysctl.conf

  1. net.ipv4.ip_local_port_range = 1025 65000
  2.  
  3. net.ipv4.tcp_tw_reuse = 1
  4. net.ipv4.tcp_tw_recycle = 1
  5.  
  6. net.ipv4.tcp_fin_timeout = 15
  7. net.ipv4.tcp_max_tw_buckets = 35000

sysctl -p

使用nginx反向代理haprox后台

  1. #省略
  2.  
  3. listen  admin_status
  4.     mode  http
  5.     bind 192.168.0.107:8000
  6.     option httplog
  7.     log global
  8.     stats enable
  9.     stats refresh 30s
  10.     stats hide-version
  11.     stats realm Haproxy\ Statistics
  12.     #stats uri  /admin-status
  13.     stats uri  /haproxy/
  14.     #stats auth  admin:123456
  15.     #stats admin if TRUE

nginx.conf

  1. #省略
  2.              location ~* ^/haproxy/
  3.              {
  4.   proxy_pass http://192.168.0.107:8000;
  5.   proxy_set_header Host $host;
  6.   proxy_set_header X-Real-IP $remote_addr;
  7.   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  8.   #proxy_set_header X-Forwarded-For $remote_addr;
  9.   proxy_redirect off;
  10.              }
  11. #省略

参考:
http://linux.die.net/man/5/xinetd.conf
http://adslroot.blogspot.com/2013/12/haproxy-mysql.html
http://sssslide.com/www.slideshare.net/Severalnines/haproxy-mysql-slides

Posted in haproxy/Atlas, 技术.

Tagged with , , , .


linux查看和改变网卡工作速率

同一机柜其它机器都在千兆模式但有几台却是百兆,调整速度后还自动降速到百兆.
最后让机房换了网线立马解决问题,数据库的进程排队也降低了

查看网卡信息,网卡支持千兆但工作在百兆.
ethtool eth2

  1. Advertised link modes: 10baseT/Half 10baseT/Full
  2.                                 100baseT/Half 100baseT/Full
  3.                                 1000baseT/Full
  4.         Advertised pause frame use: No
  5.         Advertised auto-negotiation: Yes
  6.         Speed: 100Mb/s
  7.         Duplex: Full

调整到千兆
ethtool -s eth2 speed 1000 duplex full

tail /var/log/messages

  1. Oct 23 10:17:22  C1g kernel: e1000e: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
  2. Oct 23 10:17:23  C1g kernel: e1000e: eth2 NIC Link is Down
  3. Oct 23 10:17:33  C1g kernel: e1000e: eth2 NIC Link is Up 100 Mbps Full Duplex, Flow Control: None
  4. Oct 23 10:17:33  C1g kernel: 0000:03:00.1: eth2: 10/100 speed: disabling TSO

又变回到百兆

ethtool备注
ethtool ethX //查询ethX网口基本设置
ethtool –h //显示ethtool的命令帮助(help)
ethtool –i ethX //查询ethX网口的相关信息
ethtool –d ethX //查询ethX网口注册性信息
ethtool –r ethX //重置ethX网口到自适应模式
ethtool –S ethX //查询ethX网口收发包统计
ethtool –s ethX [speed 10|100|1000]\ //设置网口速率10/100/1000M
[duplex half|full]\ //设置网口半/全双工
[autoneg on|off]\ //设置网口是否自协商

Posted in linux 维护优化.

Tagged with , , .


禁止微软搜索蜘蛛

禁止微软蜘蛛,爬的太疯狂了,还不带流量…
同时降低频率到60秒间隔.
在web根目录下编辑robots.txt

  1. User-agent: Bingbot
  2. Disallow: /
  3. User-agent: Adidxbot
  4. Disallow: /
  5. User-agent: MSNBot
  6. Disallow: /
  7. User-agent: BingPreview
  8. Disallow: /
  9. User-agent: *
  10. Disallow:
  11. Crawl-delay: 60
  12. Disallow: /api/
  13. Disallow: /data/

参考:
http://www.bing.com/webmaster/help/which-crawlers-does-bing-use-8c184ec0
http://tool.chinaz.com/robots/

Posted in 网站建设, SEO.

Tagged with , , .