04/4/14

haproxy+keepalived+mysql+xinetd 实现 mysql 双机机制

1、功能描述

a、192.168.0.103 (以下称103,vip:192.168.0.123),192.168.0.104(以下称104,vip:192.168.0.124)分别部署haproxy,并通过keepalived绑定vip(103->123,104->124)。

b、程序代码通过vip连接数据库,103、104各部署一套slave,104从103slave同步。

c、当103宕机,123ip迁移到104。此时104同时有123、124两个vip,并响应123和124数据库请求。同理,104宕机时,103同时为vip123,124提供服务。

d、haproxy通过xinetd服务,启动mysqlchk服务。数据库正常返回http 200,异常则返回503.

e、优点:一、程序迁移数据库改动比较少,运维可完全控制。二、运维可根据情况,安排调节slave使用权重,合理配置资源。三、高可用行,大大减少了单点故障影响。

2、部署过程

a、haproxy.cfg

global
log 127.0.0.1 local3 notice
maxconn 20480
uid mysql
gid mysql
daemon
nbproc 1
pidfile /opt/phpdba/haproxy/logs/haproxy.pid

defaults
option  dontlognull
retries 3
option  redispatch
#maxconn 20480
contimeout      5000
clitimeout      50000
srvtimeout      50000

frontend db_baobao_read
bind :33080
default_backend cluster_db_baobao_read

frontend db_baobao_write
bind :33081
default_backend cluster_db_baobao_write

frontend db_user_write
bind :8066
default_backend cluster_db_user_write

frontend web_haproxy
bind :80
default_backend web_status

backend cluster_db_baobao_read
mode tcp
option tcpka
balance roundrobin
option httpchk
server db103_slave 192.168.0.103:3316 weight 1 check port 8890 inter 1s rise 5 fall 3
server db104_slave 192.168.0.104:3316 weight 1 check port 8890 inter 1s rise 5 fall 3

backend cluster_db_baobao_write
mode tcp
option tcpka
balance roundrobin
option httpchk
server db110_master 192.168.0.110:3308 weight 1 check port 8890 inter 1s rise 5 fall 3

backend cluster_db_user_write
mode tcp
option tcpka
balance roundrobin
option httpchk
server db46_master 192.168.0.46:3306 weight 1 check port 8890 inter 1s rise 5 fall 3

backend web_status
mode http
stats enable
#stats scope
stats hide-version
stats refresh 5s
stats uri /status
stats realm Haproxy\ statistic
stats auth chen-123:phpdba

 

b、keepalvied_db.conf

! Configuration File for keepalived
global_defs {
router_id LVS_db_ci104
}

vrrp_script chk_http_port {
script “/opt/phpdba/keepalived/etc/keepalived/check_haproxy.sh”
interval 2
weight -10
}

vrrp_instance VI_CiUser {
state BACKUP
interface em2
virtual_router_id 58
priority 99
advert_int 1

authentication {
auth_type PASS
auth_pass phpdba
}

virtual_ipaddress {
192.168.0.123 dev em2 label em2:0
}
}

vrrp_instance VI_CiBlog_QQ {
state MASTER
interface em2
virtual_router_id 68
priority 110
advert_int 1
authentication {
auth_type PASS
auth_pass phpdba
}

track_script {
chk_http_port
}

virtual_ipaddress {
192.168.0.124 dev em2 label em2:1
}
}

c、check_haproxy.sh

#!/bin/bash
A=`ps -C haproxy –no-header |wc -l`
if [ $A -eq 0 ];then
/opt/phpdba/haproxy/sbin/haproxy -f /opt/phpdba/haproxy/conf/haproxy.cfg
sleep 3
if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
/etc/init.d/keepalived stop
fi
fi

d、/etc/xinetd.d/mysqlchk

service mysqlcheck
{
flags           = REUSE
socket_type     = stream
port            = 8890
wait            = no
user            = root
server          = /opt/phpdba/keepalived/etc/keepalived/mysqlchk.sh
log_on_failure  += USERID
disable         = no
only_from       = 192.168.0.0/24
}

e、/opt/phpdba/keepalived/etc/keepalived/mysqlchk.sh

#!/bin/bash

MYSQL_HOST=”192.168.0.104″
MYSQL_PORT=”3316″
MYSQL_USERNAME=”admin”
MYSQL_PASSWORD=”123456″
MYSQL_MAXTHREADNUM=”200″

ERROR_MSG=`/opt/phpdba/mysql/bin/mysql -h $MYSQL_HOST -P $MYSQL_PORT -u $MYSQL_USERNAME -p$MYSQL_PASSWORD -e “show databases;” 2>/dev/null`

if [ "$ERROR_MSG" != "" ]
then
/bin/echo -e “HTTP/1.1 200 OK\r\n”
/bin/echo -e “Content-Type: Content-Type: text/plain\r\n”
/bin/echo -e “\r\n”
/bin/echo -e “MySQL is running. \r\n”
/bin/echo -e “\r\n”
else
/bin/echo -e “HTTP/1.1 503 Service Unavailable\r\n”
/bin/echo -e “Content-Type: Content-Type: text/plain\r\n”
/bin/echo -e “\r\n”
/bin/echo -e “MySQL is down. \r\n”
/bin/echo -e “\r\n”
fi

f、/etc/services

mysqlcheck      8890/tcp
mysqlcheck      8890/udp

g、xinetd

yum install xinetd -f   &&  /etc/init.d/xinetd start

h、keepalvied

/etc/init.d/keepalived start

3、遇到的问题

a、haproxy 配置中check port 同一ip同一端口,只能监控一次。同一ip多个mysql实例监控,采用不同端口监控。

b、防火墙禁止了8890端口访问,导致check port失败。(iptables -I INPUT -p tcp –dport 8890 -j ACCEPT)

4、haproxy 状态页面

haproxy

 

10/31/13

HAProxy+keepalived MySQL负载均衡

环境:vip 192.168.0.233

mysql1  192.168.0.227:3306

mysql2  192.168.0.230:3306

1. 监控脚本编写(mysqlrep_status.sh)

#!/bin/bash
#
# /opt/phpdba/keepalived/etc/keepalived/mysqlrep_status.sh
#
# This script checks if a mysql server is healthy running on localhost. It will return:
#
# "HTTP/1.x 200 OK\r" (if mysql is running smoothly)
#
# – OR –
#
# "HTTP/1.x 503 Internal Server Error\r" (else)
#

MYSQL_HOST="127.0.0.1"
MYSQL_PORT="3306"
MYSQL_USERNAME="root"
MYSQL_PASSWORD="123456"
MYSQL_MAXTHREADNUM="30"

# We perform a simple query that should return a few results
/opt/phpdba/mysql/bin/mysql -h $MYSQL_HOST -P $MYSQL_PORT -u $MYSQL_USERNAME -p$MYSQL_PASSWORD -e "show full processlist;" >/tmp/processlist.txt
/opt/phpdba/mysql/bin/mysql -h $MYSQL_HOST -P $MYSQL_PORT -u $MYSQL_USERNAME -p$MYSQL_PASSWORD -e "show slave status\G;" >/tmp/rep.txt
iostat=`grep "Slave_IO_Running" /tmp/rep.txt  |awk '{print $2}'`
sqlstat=`grep "Slave_SQL_Running" /tmp/rep.txt |awk '{print $2}'`
result=$(cat /tmp/processlist.txt|wc -l)

#echo processlist num:$result iostat:$iostat and sqlstat:$sqlstat
# if slave_IO_Running and Slave_sql_Running ok,then return 200 code
if [ "$result" -lt "$MYSQL_MAXTHREADNUM" ] && [ "$iostat" = "Yes" ] && [ "$sqlstat" = "Yes" ];

then
        # mysql is fine, return http 200
        /bin/echo -e "HTTP/1.1 200 OK\r\n"

else
        # mysql is down, return http 503
        /bin/echo -e "HTTP/1.1 503 Service Unavailable\r\n"

fi

 

2、系统监听服务

a、修改/etc/service文件,

mysqlcheck      8890/tcp                # Mysql Check
mysqlcheck      8890/udp                # Mysql Check

b、在/etc/xinetd.d/目录下添加mysqlchk

service mysqlcheck
{
flags           = REUSE
socket_type     = stream
port            = 8890
wait            = no
user            = root
server          = /opt/phpdba/keepalived/etc/keepalived/mysqlrep_status.sh
log_on_failure  += USERID
disable         = no
}

或者 /etc/xinetd.conf 添加

mysqlcheck stream tcp nowait root /opt/phpdba/keepalived/etc/keepalived/mysqlrep_status.sh mysqlcheck

c、重启xinetd (如无此服务,yum -y install xinetd)

/etc/init.d/xinetd restart   or service xinetd restart

d、telnet 测试

telnet 192.168.0.233 8890
Trying 192.168.0.233…
Connected to 192.168.0.233.
Escape character is ‘^]’.
HTTP/1.1 200 OK

Connection closed by foreign host.

3、haproxy配置

global
        log 127.0.0.1 local3 notice
        maxconn 20480
        uid mysql
        gid mysql
        daemon
        nbproc 1
        pidfile /opt/phpdba/haproxy/logs/haproxy.pid

defaults
        log     global
        mode    http
        #option httplog
        option  dontlognull
        retries 3
        option  redispatch
        maxconn 20480
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000

listen  MYSQL_SLAVE  0.0.0.0:3333
        #cookie SERVERID rewrite
        mode tcp
        maxconn 20480
        balance roundrobin
        option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www
        server  mysql_227 192.168.0.227:3306  check port 8890 inter 5s rise 2 fall 3
        server  mysql_230 192.168.0.230:3306  check port 8890 inter 5s rise 2 fall 3
        srvtimeout      20000

listen  admin_status
        mode  http
        bind 0.0.0.0:8899
        option httplog
        log global
        stats enable
        stats refresh 10s
        stats hide-version
        stats realm Haproxy\ Statistics
        stats uri  /admin-status
        stats auth  admin:123456
        stats admin if TRUE

4、keepalived.conf

! Configuration File for keepalived

vrrp_script chk_http_port {
        script "/opt/phpdba/keepalived/etc/keepalived/check_haproxy.sh"
        interval 2
        weight 2

        global_defs {
                router_id LVS_db_phpdba
        }

        vrrp_instance VI_1 {
                state MASTER
                interface em1
                virtual_router_id 51
                priority 150
                advert_int 1
                authentication {
                        auth_type PASS
                        auth_pass phpdba
                }

                track_script {
                        chk_http_port
                }

                virtual_ipaddress {
                        192.168.0.233
                }
        }
}

5、haproxy进程监控(check_haproxy.sh)

#!/bin/bash
A=`ps -C haproxy --no-header |wc -l`
if [ $A -eq 0 ];then
        /opt/phpdba/haproxy/sbin/haproxy -f /opt/phpdba/haproxy/conf/haproxy.cfg
        sleep 3
        if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
                /etc/init.d/keepalived stop
        fi
fi

6、haproxy 启动脚本

cat /etc/init.d/haproxyd

#! /bin/sh
set -e

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/haproxy/sbin
PROGDIR=/opt/phpdba/haproxy
PROGNAME=haproxy
DAEMON=$PROGDIR/sbin/$PROGNAME
CONFIG=$PROGDIR/conf/$PROGNAME.conf
PIDFILE=$PROGDIR/run/$PROGNAME.pid
DESC="HAProxy daemon"
SCRIPTNAME=/etc/init.d/$PROGNAME

# Gracefully exit if the package has been removed.
test -x $DAEMON || exit 0

start()
{
        echo -n "Starting $DESC: $PROGNAME"
        $DAEMON -f $CONFIG
        echo "."
}

stop()
{
        echo -n "Stopping $DESC: $PROGNAME"
        haproxy_pid=cat $PIDFILE
        kill $haproxy_pid
        echo "."
}

restart()
{
        echo -n "Restarting $DESC: $PROGNAME"
        $DAEMON -f $CONFIG -p $PIDFILE -sf $(cat $PIDFILE)
        echo "."
}

case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  restart)
        restart
        ;;
  *)
        echo "Usage: $SCRIPTNAME {start|stop|restart}" >&2
        exit 1
        ;;
esac

exit 0

 

10/12/13

keepalived 安装配置

1、下载keepalived源码

wget http://www.keepalived.org/software/keepalived-1.2.8.tar.gz

2、tar zxvf keepalived-1.2.8.tar.gz

3、cd keepalived-1.2.8;./configure –prefix=/opt/phpdba/keepalived

当看到:

Keepalived configuration
————————
Keepalived version       : 1.2.8
Compiler                 : gcc
Compiler flags           : -g -O2
Extra Lib                : -lpopt -lssl -lcrypto -lcrypt  -lnl
Use IPVS Framework       : Yes
IPVS sync daemon support : Yes
IPVS use libnl           : Yes
Use VRRP Framework       : Yes
Use VRRP VMAC            : Yes
SNMP support             : No
SHA1 support             : No
Use Debug flags          : No

编译完成

4、make

当看到:

Make complete
make[1]: Leaving directory `/root/chen-123/keepalived-1.2.8/keepalived’
make -C genhash
make[1]: Entering directory `/root/chen-123/keepalived-1.2.8/genhash’
gcc -g -O2  -I/usr/src/linux/include -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o main.o main.c
gcc -g -O2  -I/usr/src/linux/include -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o sock.o sock.c
gcc -g -O2  -I/usr/src/linux/include -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o layer4.o layer4.c
gcc -g -O2  -I/usr/src/linux/include -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o http.o http.c
gcc -g -O2  -I/usr/src/linux/include -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes   -c -o ssl.o ssl.c
Building ../bin/genhash
strip ../bin/genhash

Make complete
make[1]: Leaving directory `/root/chen-123/keepalived-1.2.8/genhash’

Make complete

5、make install (安装)

make -C keepalived install
make[1]: Entering directory `/root/chen-123/keepalived-1.2.8/keepalived’
install -d /opt/phpdba/keepalived/sbin
install -m 700 ../bin/keepalived /opt/phpdba/keepalived/sbin/
install -d /opt/phpdba/keepalived/etc/rc.d/init.d
install -m 755 etc/init.d/keepalived.init /opt/phpdba/keepalived/etc/rc.d/init.d/keepalived
install -d /opt/phpdba/keepalived/etc/sysconfig
install -m 755 etc/init.d/keepalived.sysconfig /opt/phpdba/keepalived/etc/sysconfig/keepalived
install -d /opt/phpdba/keepalived/etc/keepalived/samples
install -m 644 etc/keepalived/keepalived.conf /opt/phpdba/keepalived/etc/keepalived/
install -m 644 ../doc/samples/* /opt/phpdba/keepalived/etc/keepalived/samples/
install -d /opt/phpdba/keepalived/share/man/man5
install -d /opt/phpdba/keepalived/share/man/man8
install -m 644 ../doc/man/man5/keepalived.conf.5 /opt/phpdba/keepalived/share/man/man5
install -m 644 ../doc/man/man8/keepalived.8 /opt/phpdba/keepalived/share/man/man8
make[1]: Leaving directory `/root/chen-123/keepalived-1.2.8/keepalived’
make -C genhash install
make[1]: Entering directory `/root/chen-123/keepalived-1.2.8/genhash’
install -d /opt/phpdba/keepalived/bin
install -m 755 ../bin/genhash /opt/phpdba/keepalived/bin/
install -d /opt/phpdba/keepalived/share/man/man1
install -m 644 ../doc/man/man1/genhash.1 /opt/phpdba/keepalived/share/man/man1
make[1]: Leaving directory `/root/chen-123/keepalived-1.2.8/genhash’

安装完成

6、配置keepalived

a、建立服务启动脚本

cp /opt/phpdba/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/keepalived

chmod +x /etc/init.d/keepalived

b、修改/etc/init.d/keepalived, 寻找大约15行左右的. /etc/sysconfig/keepalived, 修改为:
.  /opt/phpdba/keepalived/etc/sysconfig/keepalived, 即指向正确的文件位置.

或者将/opt/phpdba/keepalived/etc/sysconfig/keepalived copy 到/etc/sysconfig/ 下 。

同时在上述行下添加以下内容(将keepavlied主程序所在路径导入到环境变量PATH中):

PATH=”$PATH:/opt/phpdba/keepalived/sbin”

export PATH

c、修改/opt/phpdba/keepalived/etc/sysconfig/keepalived或者/etc/sysconfig/keepalived文件,设置正确的服务启动参数
KEEPALIVED_OPTIONS=”-D -f /opt/phpdba/keepalived/etc/keepalived/keepalived.conf”

d、经过以上修改,keepalived基本安装即可完成。

启动方式:service keepalived start  or /etc/init.d/keepalived start

e、将此服务设置为开机启动:chkconfig keepalived on

7、编写keepalived配置文件

! Configuration File for keepalived
# author by chen-123@phpdba.com

global_defs {
notification_email {
chen-123@phpdba.com
}
notification_email_from keepalived@phpdba.com
smtp_server smtp.163.com
smtp_connect_timeout 30
router_id LVS_1
}

vrrp_instance phpdba_1 {
state MASTER
interface em1
virtual_router_id 50
nopreempt
priority 100
advert_int 1
authentication{
auth_type PASS
auth_pass phpdba
}

virtual_ipaddress {
192.168.0.226
192.168.0.233
}
}

virtual_server 192.168.0.226 80 { #设置虚拟服务器,需要指定虚拟ip地址和服务端口,ip与端口之间用空格隔开
delay_loop 6 #设置健康检查时间,单位是秒
lb_algo rr #设置负载调度算法,这里设置为rr,即轮询算法
lb_kind DR #设置LVS实现负载均衡的机制,可以有NAT、TUN和DR三个模式可选
persistence_timeout 50 #会话保持时间,单位是秒,这个选项对于动态网页是非常有用的,为集群系统中session共享提供了一个很好的解决方案。有了这个会话保持功能,用户的请求会被一直分发到某个服务节点,直到超过
这个会话保持时间。需要注意的是,这个会话保持时间,是最大无响应超时时间,也就是说用户在操作动态页面时,如果在50秒内没有执行任何操作,那么接下来的操作会被分发到另外节点,但是如果一直在操作动态页面,则不受50秒的时间限制
protocol TCP

#sorry_server 192.168.200.200 1358

real_server 192.168.0.227 80 { #配置服务节点1,需要指定real server的真实IP地址和端口,ip与端口之间用空格隔开
weight 1 #配置服务节点的权值,权值大小用数字表示,数字越大,权值越高,设置权值的大小可以为不同性能的服务器分配不同的负载,可以对性能高的服务器设置较高的权值,而对性能较低的服务器设置相对较低的权>值,这样就合理的利用和分配了系统资源
#HTTP_GET {
#    url {
#      path /testurl3/test.jsp
#      digest 640205b7b0fc66c1ea91c463fac6334d
#    }
#    connect_timeout 3
#    nb_get_retry 3
#    delay_before_retry 3
#}
TCP_CHECK {             #realserve的状态检测设置部分,单位是秒
connect_timeout 10     #10秒无响应超时
nb_get_retry 3              #重试次数
delay_before_retry 3    #重试间隔
}
}

real_server 192.168.0.228 80 {
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
}
}

}

 

8、绑定ip

ifconfig em1:0 192.168.0.226 netmask 255.255.255.0 up

ifconfig em1:1 192.168.0.233 netmask 255.255.255.0 up

route add -host 192.168.0.226 dev em1:0

route add -host 192.168.0.233 dev em1:1

echo “1″>/proc/sys/net/ipv4/ip_forward

9、配置ipvs

[root@localhost ~]# ipvsadm -C
[root@localhost ~]# ipvsadm -A -t 192.168.0.226:80 -s rr -p 600
[root@localhost ~]# ipvsadm -a -t 192.168.0.226:80 -r 192.168.0.227:80 -g
[root@localhost ~]# ipvsadm -a -t 192.168.0.226:80 -r 192.168.0.228:80 -g
[root@localhost ~]# ipvsadm -A -t 192.168.0.233:80 -s rr -p 600
[root@localhost ~]# ipvsadm -a -t 192.168.0.233:80 -r 192.168.0.227:80 -g
[root@localhost ~]# ipvsadm -a -t 192.168.0.233:80 -r 192.168.0.228:80 -g

[root@localhost ~l]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.226:http rr persistent 600
-> 192.168.0.227:http           Route   1      0          0
-> 192.168.0.228:http           Local   1      0          0
TCP  192.168.0.233:http rr persistent 600
-> 192.168.0.227:http           Route   1      0          0
-> 192.168.0.228:http           Local   1      0          0

lvs_ds脚本化:

#!/bin/sh
# description: Start LVS of Director server
# add by chen-123 (未对参数进行验证)
VIP=$1
RIP1=$2
RIP2=$3
./etc/rc.d/init.d/functions
case “$1″ in
start)
echo ” start LVS of Director Server”
# set the Virtual  IP Address and sysctl parameter
/sbin/ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 up
echo “1″ >/proc/sys/net/ipv4/ip_forward
#Clear IPVS table
/sbin/ipvsadm -C
#set LVS
/sbin/ipvsadm -A -t $VIP:80 -s rr -p 600
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
#Run LVS
/sbin/ipvsadm
;;
stop)
echo “close LVS Directorserver”
echo “0″ >/proc/sys/net/ipv4/ip_forward
/sbin/ipvsadm -C
/sbin/ifconfig eth0:0 down
;;
*)
echo “Usage: $0 {start|stop}”
exit 1
esac

脚本使用:sh lvs_ds.sh 192.168.0.226(233)  192.168.0.227 192.168.0.228

10、由于虚拟ip,也就是上面的VIP地址,是Director Server和所有的Real server共享的。

如果有ARP请求VIP地址时,Director Server与所有Real server都做应答的话,就出现问题了。因此,需要禁止Real server响应ARP请求。

而lvs_lo.sh脚本的作用就是使Real Server不响应arp请求。

lvs_lo脚本:

#!/bin/bash
#在回环设备上绑定虚拟ip地址,设置子网掩码255.255.255.255,与Director Server 上的虚拟ip互通,然后禁止本机arp请求。
VIP=$1
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
echo “1″ >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo “2″ >/proc/sys/net/ipv4/conf/lo/arp_announce
echo “1″ >/proc/sys/net/ipv4/conf/all/arp_ignore
echo “2″ >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p
#end

使用:sh lvs_lo.sh 192.168.0.226(233)

11、启动keepalived 服务

/etc/init.d/keepalived start

tail -f /var/log/message

Oct 12 16:33:00 localhost Keepalived[40106]: Starting Keepalived v1.2.8 (10/12,2013)
Oct 12 16:33:00 localhost Keepalived[40107]: Starting Healthcheck child process, pid=40108
Oct 12 16:33:00 localhost Keepalived[40107]: Starting VRRP child process, pid=40109
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Netlink reflector reports IP 192.168.0.228 added
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Netlink reflector reports IP 192.168.0.226 added
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Netlink reflector reports IP 192.168.0.233 added
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Netlink reflector reports IP fe80::f21f:afff:fedc:e9dd added
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Registering Kernel netlink reflector
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Netlink reflector reports IP 192.168.0.228 added
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Registering Kernel netlink command channel
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Netlink reflector reports IP 192.168.0.226 added
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Registering gratuitous ARP shared channel
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Netlink reflector reports IP 192.168.0.233 added
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Netlink reflector reports IP fe80::f21f:afff:fedc:e9dd added
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Registering Kernel netlink reflector
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Registering Kernel netlink command channel
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Opening file ‘/opt/phpdba/keepalived/etc/keepalived/keepalived.conf’.
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Configuration is using : 14131 Bytes
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Opening file ‘/opt/phpdba/keepalived/etc/keepalived/keepalived.conf’.
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Configuration is using : 61003 Bytes
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: Using LinkWatch kernel netlink reflector…
Oct 12 16:33:00 localhost Keepalived_vrrp[40109]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: IPVS: Service already exists
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: IPVS: Destination already exists
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: IPVS: Destination already exists
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Using LinkWatch kernel netlink reflector…
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Activating healthchecker for service [192.168.0.227]:80
Oct 12 16:33:00 localhost Keepalived_healthcheckers[40108]: Activating healthchecker for service [192.168.0.228]:80
Oct 12 16:33:01 localhost Keepalived_vrrp[40109]: VRRP_Instance(phpdba_1) Transition to MASTER STATE
Oct 12 16:33:02 localhost Keepalived_vrrp[40109]: VRRP_Instance(phpdba_1) Entering MASTER STATE

启动完成!

[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether f0:1f:af:dc:e9:dd brd ff:ff:ff:ff:ff:ff
inet 192.168.0.228/24 brd 192.168.0.255 scope global em1
inet 192.168.0.226/24 brd 192.168.0.255 scope global secondary em1:0
inet 192.168.0.233/24 brd 192.168.0.255 scope global secondary em1:1
inet6 fe80::f21f:afff:fedc:e9dd/64 scope link
valid_lft forever preferred_lft forever
3: em2: <BROADCAST,MULTICAST> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether f0:1f:af:dc:e9:de brd ff:ff:ff:ff:ff:ff

12、启动其他节点

Real server上执行如下脚本:
[root@rs*~]#sh lvs_lo.sh 192.168.0.226(233)
至此,Keepalived+lvs高可用的LVS集群系统已经运行起来了。