操作系统

当前位置:金沙棋牌 > 操作系统 > keepalived负载均衡实战,LVS实现高可用集群

keepalived负载均衡实战,LVS实现高可用集群

来源:http://www.logblo.com 作者:金沙棋牌 时间:2019-09-03 16:41

一、需求背景

一、需求背景

操作系统平台:CentOS5.2
软件:LVS keepalived
LVS Keepalived 介绍

LVS keepalived负载均衡实战

发现公司好多项目都用到memched。并且都是单点,memched服务器 挂了就会影响业务,于是百度之,发现memched不可以集群,也没有看到有什么高可用方案,但是发现了repcached(memched主从复制),结合LVS的NAT模式不同端口转发实现memched双机HA,本文章记录本人的部署过程,从属实战,没有文字性的说明,欢迎各位指点。

发现公司好多项目都用到memched。并且都是单点,memched服务器 挂了就会影响业务,于是百度之,发现memched不可以集群,也没有看到有什么高可用方案,但是发现了repcached(memched主从复制),结合LVS的NAT模式不同端口转发实现memched双机HA,本次文章记录本人的部署过程,从属实战,没有文件性的说明,不过大牛们一看就懂了。
 
二、环境:
 
1、系统环境:CentOS release 6.4 (Final)
 
2、网络环境
 
调度机(master):
 
vip:172.28.26.100
 
vip1:172.28.16.100(lvs转发网关  )
 
eth1:172.28.26.101 (内网)
 
eth2:172.28.16.101(lvs转发网段)
 
调度机(backup):
 
eth1:172.28.26.99 (内网)
 
eth2:172.28.16.99(lvs转发网段)
 
memched备节点:
 
eth1:172.28.26.102 (内网)
 
eth2:172.28.16.102(lvs转发网段)
 
memched主节点:  

**LVS

       ---------------------这里非常感谢田逸老师的文档,在我学习lvs的时候给了我很大的帮助!

二、环境:

eth1:172.28.26.103 (内网)
 
eth2:172.28.16.103(lvs转发网段)
 
3、路由策略
 
调度机(master):
 
echo '201      eth1' >> /etc/iproute2/rt_tables
 
echo 'default table eth1 via 172.28.26.1 dev eth1' > /etc/sysconfig/network-scripts/route-eth1
 
echo 'from 172.28.26.101/255.255.255.255 table eth1' > /etc/sysconfig/network-scripts/rule-eth1
 
echo 'from 172.28.26.100/255.255.255.255 table eth1' >> /etc/sysconfig/network-scripts/rule-eth1
 
echo 'from 172.28.16.101/32 table ZW_LVS_LAN' > /etc/sysconfig/network-scripts/rule-eth2
 
memched节点1:
 
echo 'default table ZW_LVS_LAN via 172.28.16.100' > /etc/sysconfig/network-scripts/route-eth2
 echo 'from 172.28.16.0/24 table ZW_LVS_LAN' > /etc/sysconfig/network-scripts/rule-eth2
 echo '202      ZW_LVS_LAN' >> /etc/iproute2/rt_tables
 
memched节点2:
 
echo 'default table ZW_LVS_LAN via 172.28.16.100' > /etc/sysconfig/network-scripts/route-eth2
 echo 'from 172.28.16.0/24 table ZW_LVS_LAN' > /etc/sysconfig/network-scripts/rule-eth2
 echo '202      ZW_LVS_LAN' >> /etc/iproute2/rt_tables
 
4、内核参数
 fs.file-max = 1000000
 kernel.core_uses_pid = 1
 kernel.msgmax = 1048560
 kernel.msgmnb = 1073741824
 kernel.shmall = 4294967296
 kernel.shmmax = 68719476736
 kernel.sysrq = 0
 net.core.netdev_max_backlog = 1048576
 net.core.rmem_default = 2097152
 net.core.rmem_max = 16777216
 net.core.somaxconn = 1048576
 net.core.wmem_default = 2097152
 net.core.wmem_max = 16777216
 net.ipv4.conf.default.accept_source_route = 0
 net.ipv4.conf.default.rp_filter = 1
 net.ipv4.ip_forward = 1
 net.ipv4.ip_local_port_range = 1024    65000
 net.ipv4.neigh.default.gc_thresh1 = 10240
 net.ipv4.neigh.default.gc_thresh2 = 40960
 net.ipv4.neigh.default.gc_thresh3 = 81920
 net.ipv4.tcp_fin_timeout = 1
 net.ipv4.tcp_keepalive_intvl = 15
 net.ipv4.tcp_keepalive_probes = 5
 net.ipv4.tcp_keepalive_time = 30
 net.ipv4.tcp_max_orphans = 3276800
 net.ipv4.tcp_max_syn_backlog = 1048576
 net.ipv4.tcp_max_tw_buckets = 50000
 net.ipv4.tcp_mem = 94500000 915000000 927000000
 net.ipv4.tcp_orphan_retries = 3
 net.ipv4.tcp_reordering = 5
 net.ipv4.tcp_retrans_collapse = 0
 net.ipv4.tcp_retries2 = 5
 net.ipv4.tcp_rmem = 4096        87380  4194304
 net.ipv4.tcp_sack = 1
 net.ipv4.tcp_synack_retries = 1
 net.ipv4.tcp_syncookies = 0
 net.ipv4.tcp_syn_retries = 1
 net.ipv4.tcp_timestamps = 1
 net.ipv4.tcp_tw_recycle = 1
 net.ipv4.tcp_tw_reuse = 1
 net.ipv4.tcp_window_scaling = 1
 net.ipv4.tcp_wmem = 4096        16384  4194304
 net.ipv6.conf.all.disable_ipv6 = 1
 net.ipv6.conf.default.disable_ipv6 = 1
 net.ipv4.conf.eth0.rp_filter = 1 #memched节点不需要
 net.ipv4.conf.eth1.rp_filter = 1 #memched节点不需要
 net.ipv4.conf.eth2.rp_filter = 1 #memched节点不需要
net.ipv4.conf.all.rp_filter = 0    #memched节点不需要

LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立,是中国国内最早出现的自由软件项目之一。目前有三种IP负载均衡技术(VS/NAT、VS/TUN和VS/DR);
十种调度算法(rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq)。
**Keepalvied
Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现

一、为什么要使用负载均衡技术?

1、系统环境:

图片 1

图片 2

1、系统高可用性

  1. CentOSrelease6.4(Final)

**IP配置信息:

2、  系统可扩展性

2、网络环境


3、  负载均衡能力

  1. 调度机(master):
  2. vip:172.28.26.100
  3. vip1:172.28.16.100(lvs转发网关)
  4. eth1:172.28.26.101(内网)
  5. eth2:172.28.16.101(lvs转发网段)
  6. 调度机(backup):
  7. eth1:172.28.26.99(内网)
  8. eth2:172.28.16.99(lvs转发网段)
  9. memched备节点:
  10. eth1:172.28.26.102(内网)
  11. eth2:172.28.16.102(lvs转发网段)
  12. memched主节点:
  13. eth1:172.28.26.103(内网)
  14. eth2:172.28.16.103(lvs转发网段)
  • LVS-DR-Master 192.168.2.166

  • LVS-DR-BACKUP 192.168.2.167

  • LVS-DR-VIP 192.168.2.170

  • WEB1-Realserver 192.168.2.171

  • WEB2-Realserver 192.168.2.172

  • GateWay 192.168.2.253
    安装LVS和Keepalvied软件包

    1. 下载相关软件包
      #mkdir /usr/local/src/lvs
      #cd /usr/local/src/lvs
      #wget

      #wget

    2. 安装LVS和Keepalived

  • #lsmod |grep ip_vs

  • #uname -r

  • 2.6.18-53.el5PAE

  • #ln -s /usr/src/kernels/2.6.18-53.el5PAE-i686/ /usr/src/linux

  • #tar zxvf ipvsadm-1.24.tar.gz

  • #cd ipvsadm-1.24

  • #make && make install

  • #find / -name ipvsadm # 查看ipvsadm的位置

  • #tar zxvf keepalived-1.1.15.tar.gz

  • #cd keepalived-1.1.15

  • #./configure && make && make install

  • #find / -name keepalived # 查看keepalived位置

  • #cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/

  • #cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

  • #mkdir /etc/keepalived

  • #cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

  • #cp /usr/local/sbin/keepalived /usr/sbin/

  • #service keepalived start|stop #做成系统启动服务方便管理.
    四. 配置LVS实现负载均衡

    1. LVS-DR,配置LVS脚本实现负载均衡

    vi /usr/local/sbin/lvs-dr.sh

    #!/bin/bash
    # description: start LVS of DirectorServer
    #Written by :NetSeek

    GW=192.168.2.253

    # website director vip.
    WEB_VIP=192.168.2.170
    WEB_RIP1=192.168.2.171
    WEB_RIP2=192.168.2.172
    . /etc/rc.d/init.d/functions

    logger $0 called with $1

    case "$1" in

    start)
    # Clear all iptables rules.
    /sbin/iptables -F
    # Reset iptables counters.
    /sbin/iptables -Z
    # Clear all ipvsadm rules/services.
    /sbin/ipvsadm -C

    #set lvs vip for dr
    /sbin/ipvsadm --set 30 5 60
    /sbin/ifconfig eth0:0 $WEB_VIP broadcast $WEB_VIP netmask 255.255.255.255 up
    /sbin/route add -host $WEB_VIP dev eth0:0
    /sbin/ipvsadm -A -t $WEB_VIP:80 -s wrr -p 3
    /sbin/ipvsadm -a -t $WEB_VIP:80 -r $WEB_RIP1:80 -g -w 1
    /sbin/ipvsadm -a -t $WEB_VIP:80 -r $WEB_RIP2:80 -g -w 1
    touch /var/lock/subsys/ipvsadm >/dev/null 2>&1

    # set Arp
    /sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW >/dev/null 2>&1
    ;;
    stop)
    /sbin/ipvsadm -C
    /sbin/ipvsadm -Z
    ifconfig eth0:0 down
    route del $WEB_VIP >/dev/null 2>&1
    rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1
    /sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW
    echo "ipvsadm stoped"
    ;;

    status)

    if [ ! -e /var/lock/subsys/ipvsadm ];then
    echo "ipvsadm is stoped"
    exit 1
    else
    ipvsadm -ln
    echo "..........ipvsadm is OK."
    fi
    ;;

    *)
    echo "Usage: $0 {start|stop|status}"
    exit 1
    esac

    exit 0
    2. 配置Realserver脚本.

    #!/bin/bash
    # Written by NetSeek
    # description: Config realserver lo and apply noarp
    WEB_VIP=192.168.2.170

    . /etc/rc.d/init.d/functions

    case "$1" in
    start)
    ifconfig lo:0 $WEB_VIP netmask 255.255.255.255 broadcast $WEB_VIP
    /sbin/route add -host $WEB_VIP dev lo:0
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
    sysctl -p >/dev/null 2>&1
    echo "RealServer Start OK"

    ;;
    stop)
    ifconfig lo:0 down
    route del $WEB_VIP >/dev/null 2>&1
    echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
    echo "RealServer Stoped"
    ;;
    status)
    # Status of LVS-DR real server.
    islothere=`/sbin/ifconfig lo:0 | grep $WEB_VIP`
    isrothere=`netstat -rn | grep "lo:0" | grep $web_VIP`
    if [ ! "$islothere" -o ! "isrothere" ];then
    # Either the route or the lo:0 device
    # not found.
    echo "LVS-DR real server Stopped."
    else
    echo "LVS-DR Running."
    fi
    ;;
    *)
    # Invalid entry.
    echo "$0: Usage: $0 {start|status|stop}"
    exit 1
    ;;
    esac
    exit 0

    附上realserver机上的/etc/sysctl.conf:

    # Kernel sysctl configuration file for Red Hat Linux
    #
    # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
    # sysctl.conf(5) for more details.
    # Controls IP packet forwarding
    net.ipv4.ip_forward = 1
    # Controls source route verification
    net.ipv4.conf.default.rp_filter = 1
    # Do not accept source routing
    net.ipv4.conf.default.accept_source_route = 0
    # Controls the System Request debugging functionality of the kernel
    kernel.sysrq = 0
    # Controls whether core dumps will append the PID to the core filename.
    # Useful for debugging multi-threaded applications.
    kernel.core_uses_pid = 1
    net.ipv4.conf.lo.arp_ignore = 1
    net.ipv4.conf.lo.arp_announce = 2
    net.ipv4.conf.all.arp_ignore = 1
    net.ipv4.conf.all.arp_announce = 2
    或者采用secondary ip address方式配置
    # vi /etc/sysctl.conf

    添加以下内容如上所示:

    net.ipv4.conf.lo.arp_ignore = 1
    net.ipv4.conf.lo.arp_announce = 2
    net.ipv4.conf.all.arp_ignore = 1
    net.ipv4.conf.all.arp_announce = 2
    #sysctl –p
    #ip addr add 61.164.122.8/32 dev lo
    #ip add list 查看是否绑定

    1. 启动lvs-dr脚本和realserver启本,在DR上可以查看LVS当前状态:
      #watch ipvsadm –ln

    五.利用Keepalvied实现负载均衡和和高可用性

    1.配置在主负载均衡服务器上配置keepalived.conf
    #vi /etc/keepalived/keepalived.conf (主调度器)
    ! Configuration File for keepalived
    global_defs {
    notification_email {

    [email protected]

    [email protected]

    [email protected]
    }
    notification_email_from
    [email protected]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id LVS_DEVEL
    }
    vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
    auth_type PASS
    auth_pass 1111
    }
    virtual_ipaddress {
    192.168.2.170
    }
    }
    virtual_server 192.168.2.170 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    persistence_timeout 60
    protocol TCP

    real_server 192.168.2.171 80 {
    weight 3
    TCP_CHECK {
    connect_timeout 10
    nb_get_retry 3
    delay_before_retry 3
    connect_port 80
    }
    }
    real_server 192.168.2.172 80 {
    weight 3
    TCP_CHECK {
    connect_timeout 10
    nb_get_retry 3
    delay_before_retry 3
    connect_port 80
    }
    }
    }

    在备用调度器上:
    #vi /etc/keepalived/keepalived.conf (备用调度器)
    ! Configuration File for keepalived
    global_defs {
    notification_email {

    [email protected]

    [email protected]

    [email protected]
    }
    notification_email_from
    [email protected]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id LVS_DEVEL
    }
    vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
    auth_type PASS
    auth_pass 1111
    }
    virtual_ipaddress {
    192.168.2.170
    }
    }
    virtual_server 192.168.2.170 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    persistence_timeout 60
    protocol TCP

    real_server 192.168.2.171 80 {
    weight 3
    TCP_CHECK {
    connect_timeout 10
    nb_get_retry 3
    delay_before_retry 3
    connect_port 80
    }
    }
    real_server 192.168.2.172 80 {
    weight 3
    TCP_CHECK {
    connect_timeout 10
    nb_get_retry 3
    delay_before_retry 3
    connect_port 80
    }
    }
    }

    2. BACKUP服务器同上配置,先安装lvs再按装keepalived,仍后配置/etc/keepalived/keepalived.conf,只需将红色标示的部分改一下即可.

    1. vi /etc/rc.local
      #/usr/local/sbin/lvs-dr.sh 将lvs-dr.sh这个脚本注释掉。
      #/usr/local/sbin/lvs-dr.sh stop 停止lvs-dr脚本
      #/etc/init.d/keepalived start 启动keepalived 服务,keepalived就能利用keepalived.conf 配
      置文件,实现负载均衡和高可用.
    2. 查看lvs服务是否正常

    #watch ipvsadm –ln
    IP Virtual Server version 1.2.1 (size=4096)
    Prot LocalAddressort Scheduler Flags
    -> RemoteAddressort Forward Weight ActiveConn InActConn
    TCP 61.164.122.8:80 wrr persistent 60
    -> 61.164.122.10:80 Route 3 0 0
    -> 61.164.122.9:80 Route 3 0 0
    复制代码
    #tail –f /var/log/message 监听日志,查看状态,测试LVS负载均衡及高可用性是否有效。
    5.停Master服务器的keepalived服务,查看BAKCUP服务器是否能正常接管服务。
    四.相关参考
    1.LVS 基础知识汇总
    LVS的算法介绍

    学习LVS的三种转发模式

    LVS中的IP负载均衡技术

    更多的请到

    负载均衡版查看
    Keepalived 相关参考资料。

LVS keepalived能很好的实现以上的要求,LVS提供负载均衡,keepalived提供健康检查,故障转移,提高系统的可用性!采用这样的架构以后很容易对现有系统进行扩展,只要在后端添加或者减少realserver,只要更改lvs的配置文件,并能实现无缝配置变更!

3、路由策略

软件:LVS keepalived LVS Keepalived 介绍 LVS LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系...

二、LVS Keepalived介绍

  1. 调度机(master):
  2. echo'201eth1'>>/etc/iproute2/rt_tables
  3. echo'defaulttableeth1via172.28.26.1deveth1'>/etc/sysconfig/network-scripts/route-eth1
  4. echo'from172.28.26.101/255.255.255.255tableeth1'>/etc/sysconfig/network-scripts/rule-eth1
  5. echo'from172.28.26.100/255.255.255.255tableeth1'>>/etc/sysconfig/network-scripts/rule-eth1
  6. echo'from172.28.16.101/32tableZW_LVS_LAN'>/etc/sysconfig/network-scripts/rule-eth2
  7. memched节点1:
  8. echo'defaulttableZW_LVS_LANvia172.28.16.100'>/etc/sysconfig/network-scripts/route-eth2
  9. echo'from172.28.16.0/24tableZW_LVS_LAN'>/etc/sysconfig/network-scripts/rule-eth2
  10. echo'202ZW_LVS_LAN'>>/etc/iproute2/rt_tables
  11. memched节点2:
  12. echo'defaulttableZW_LVS_LANvia172.28.16.100'>/etc/sysconfig/network-scripts/route-eth2
  13. echo'from172.28.16.0/24tableZW_LVS_LAN'>/etc/sysconfig/network-scripts/rule-eth2
  14. echo'202ZW_LVS_LAN'>>/etc/iproute2/rt_tables

1、  LVS

4、内核参数

LVS是一个开源的软件,可以实现LINUX平台下的简单负载均衡。LVS是Linux Virtual Server的缩写,意思是Linux虚拟服务器。目前有三种IP负载均衡技术(VS/NAT、VS/TUN和VS/DR);八种调度算法(rr,wrr,lc,wlc,lblc,lblcr,dh,sh)。

  1. fs.file-max=1000000
  2. kernel.core_uses_pid=1
  3. kernel.msgmax=1048560
  4. kernel.msgmnb=1073741824
  5. kernel.shmall=4294967296
  6. kernel.shmmax=68719476736
  7. kernel.sysrq=0
  8. net.core.netdev_max_backlog=1048576
  9. net.core.rmem_default=2097152
  10. net.core.rmem_max=16777216
  11. net.core.somaxconn=1048576
  12. net.core.wmem_default=2097152
  13. net.core.wmem_max=16777216
  14. net.ipv4.conf.default.accept_source_route=0
  15. net.ipv4.conf.default.rp_filter=1
  16. net.ipv4.ip_forward=1
  17. net.ipv4.ip_local_port_range=102465000
  18. net.ipv4.neigh.default.gc_thresh1=10240
  19. net.ipv4.neigh.default.gc_thresh2=40960
  20. net.ipv4.neigh.default.gc_thresh3=81920
  21. net.ipv4.tcp_fin_timeout=1
  22. net.ipv4.tcp_keepalive_intvl=15
  23. net.ipv4.tcp_keepalive_probes=5
  24. net.ipv4.tcp_keepalive_time=30
  25. net.ipv4.tcp_max_orphans=3276800
  26. net.ipv4.tcp_max_syn_backlog=1048576
  27. net.ipv4.tcp_max_tw_buckets=50000
  28. net.ipv4.tcp_mem=94500000915000000927000000
  29. net.ipv4.tcp_orphan_retries=3
  30. net.ipv4.tcp_reordering=5
  31. net.ipv4.tcp_retrans_collapse=0
  32. net.ipv4.tcp_retries2=5
  33. net.ipv4.tcp_rmem=4096873804194304
  34. net.ipv4.tcp_sack=1
  35. net.ipv4.tcp_synack_retries=1
  36. net.ipv4.tcp_syncookies=0
  37. net.ipv4.tcp_syn_retries=1
  38. net.ipv4.tcp_timestamps=1
  39. net.ipv4.tcp_tw_recycle=1
  40. net.ipv4.tcp_tw_reuse=1
  41. net.ipv4.tcp_window_scaling=1
  42. net.ipv4.tcp_wmem=4096163844194304
  43. net.ipv6.conf.all.disable_ipv6=1
  44. net.ipv6.conf.default.disable_ipv6=1
  45. net.ipv4.conf.eth0.rp_filter=1#memched节点不需要
  46. net.ipv4.conf.eth1.rp_filter=1#memched节点不需要
  47. net.ipv4.conf.eth2.rp_filter=1#memched节点不需要
  48. net.ipv4.conf.all.rp_filter=0#memched节点不需要

2、  keepalived

三、服务安装1、调度机(master、backup)

Keepalived 是运行在lvs 之上,它的主要功能是实现真实机的故障隔离及负载均衡器间的失败切换,提高系统的可用性

  1. yum-yinstallipvsadmkeepalivedsendmail

三、LVS keepalived负载均衡架构图

2、memched节点1、memched节点2

  图片 3

  1. wget
  2. wget
  3. wget

四、LVS keepalived的安装和配置

1.         配置环境

  1. tar-zxvflibevent-1.4.13-stable.tar.gz
  2. cdlibevent-1.4.13-stable
  3. ./configure--prefix=/usr/local
  4. make
  5. makeinstall
    1. tar-zxvfmemcached-1.4.15.tar.gz
  6. cdmemcached-1.4.15
  7. ./configure--with-libevent=/usr/local
  8. make
  9. makeinstall
    1. tar-zxvfmemcached-1.2.8-repcached-2.2.tar.gz
  10. cdmemcached-1.2.8-repcached-2.2
  11. ./configure--enable-replication
  12. make
  13. makeinstall

              System OS:CentOS release 5.4

四、keepalived配置(master、backup)[[email protected] keepalived]# cat /etc/keepalived/gobal_module

              Software:ipvsadm-1.24.tar.gz, keepalived-.tar.gz

  1. !globalconfigurefile
    1. global_defs{
  2. notification_email{
  3. [email protected]
  4. }
  5. [email protected]
  6. smtp_server127.0.0.1
  7. smtp_connect_timeout30
  8. router_idZH_DG_LVS1
  9. }
    1. [[email protected]]#cat/etc/keepalived/keepalived.conf
  10. !Keepalivedmainconfigurefile
  11. include/etc/keepalived/gobal_module
  12. include/etc/keepalived/vrrpd_module
  13. include/etc/keepalived/lvs_module

2.         信息列表

[[email protected] keepalived]# cat /etc/keepalived/vrrpd_module

名称

IP

 LVS-Master

192.168.1.112

 LVS-BACKUP

192.168.1.113

 LVS-VIP

192.168.1.115

 Realserver1

192.168.1.105

 Realserver2

192.168.1.103

Realserver3

192.168.1.104

  1. vrrp_sync_groupZH_DG_Memcached1{
  2. group{
  3. ZH_DG_WAN1
  4. ZH_DG_LAN1
  5. }
  6. smtp_alter
  7. }
    1. vrrp_instanceZH_DG_WAN1{
  8. stateMASTER(从上为:backup)
  9. interfaceeth1
  10. virtual_router_id10(从为:9)
  11. priority100
  12. advert_int1
  13. authentication{
  14. auth_typePASS
  15. auth_pass08756CD0
  16. }
  17. virtual_ipaddress{
  18. 172.28.26.100#vip
  19. }
  20. }
      1. vrrp_instanceZH_DG_LAN2{
  21. stateMASTER(从上为:backup)
  22. interfaceeth2
  23. virtual_router_id10(从为:9)
  24. priority100
  25. advert_int1
  26. authentication{
  27. auth_typePASS
  28. auth_pass6F8DBC2E
  29. }
  30. virtual_ipaddress{
  31. 172.28.16.100#Memcached虚拟网关
  32. }
  33. }

3.         安装lvs

[[email protected] keepalived]# cat /etc/keepalived/lvs_module

分别在backup lvs和master lvs上安装

  1. virtual_server172.28.26.10011211{
  2. delay_loop6
  3. lb_algowrr
  4. lb_kindNAT
  5. persistence_timeout60
  6. protocolTCP
    1. include/etc/keepalived/realserver/172.28.26.102_11234.conf
  7. include/etc/keepalived/realserver/172.28.26.103_11233.conf
  8. }
      1. [[email protected]]#cat/etc/keepalived/realserver/172.28.26.102_11234.conf
  9. real_server172.28.16.10211234{
  10. weight1#权重
  11. inhibit_on_failure#故障会修改权重为零
  12. TCP_CHECK{
  13. connect_timeout10#10秒无响应超时
  14. nb_get_retry3#重连次数
  15. delay_before_retry3#重连间隔,单位为秒
  16. connect_port11234#检测端口
  17. }
  18. }
      1. [[email protected]]#cat/etc/keepalived/realserver/172.28.26.103_11233.conf
  19. real_server172.28.16.10311233{
  20. weight1#权重
  21. inhibit_on_failure#故障会修改权重为零
  22. TCP_CHECK{
  23. connect_timeout10#10秒无响应超时
  24. nb_get_retry3#重连次数
  25. delay_before_retry3#重连间隔,单位为秒
  26. connect_port11233#检测端口
  27. }
  28. }

wget

五、启动服务

ln -s /usr/src/kernels/-164.el5-i686/ /usr/src/linux

1、keepalived启动(master和backup)

tar zxvf ipvsadm-1.24.tar.gz

  1. /etc/init.d/keepalivedstart

cd ipvsadm-1.24

memcached的启动主节点:

make && make install

  1. /usr/local/bin/memcached-d-v-l0.0.0.0-p11233-X11244-uroot(11244是数据同步端口)

4.         安装keepalived

备节点:

分别在backup lvs和master lvs上安装

  1. /usr/local/bin/memcached-d-v-l0.0.0.0-p11234-uroot-x172.28.16.103-X11244(11234是服务监听端口,11244是监听数据同步端口;-x指定masterIP,-X指定master数据同步监听端口)

wget
tar zxvf keepalived-1.1.19.tar.gz
cd keepalived-1.1.19
./configure --prefix=/usr/local/keepalived
make
make install
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived

5.         配置keepadlived

  1. PS:
  2. 1、如果启动的时候报错如下:
  3. [[email protected]_memcached_slave~]#/usr/local/bin/memcached-d-v-l0.0.0.0-p11233-X11244-uroot
  4. /usr/local/bin/memcached:errorwhileloadingsharedlibraries:libevent-1.4.so.2:cannotopensharedobjectfile:Nosuchfileordirectory
  5. 请yum安装libeven
  6. [[email protected]_memcached_slave~]#yum-yinstalllibeven
  7. 2、repcached是日本人开发的实现memcached复制功能,它是一个单master单slave的方案,但它的master/slave都是可读写的,而且可以相互同步,如果master坏掉,slave侦测到连接断了,它会自动listen而成为master;而如果slave坏掉,master也会侦测到连接断,它就会重新listen等待新的slave加入;master没有抢占功能,如果master挂掉再起来只能是从了,并且永远只能是从。

LVS-Master的配置文件如下

六、测试

[root@linux5 ~]# cat /etc/keepalived/keepalived.conf

1、写一个php测试文件

global_defs {

vi session.php

notification_email {

  1. <?php
  2. ini_set('session.save_handler','memcached');
  3. ini_set('session.save_path',"172.28.26.100:11211");
  4. session_start();
  5. $_SESSION['aa']="bb";
    1. echosession_id();
  6. ?>

               jimo291@gmail.com    #email 通知

2、用php执行这个文件会得到一个字符串,telnet172.28.26.100 11211 或者telnet172.28.26.103 11233再或者telnet172.28.26.103 11234,如果能得到值说明session已经写入memched,down到任何一台memched数据不会丢。

        }

本文出自 “为了梦想奋斗” 博客,转载请与作者联系!
原文地址:

        notification_email_from jimo291@gmail.com

发现公司好多项目都用到memched。并且都是单点,memched服务器 挂了就会影响业务,于是百度之,发现memched不可以集群,也没...

        smtp_server 127.0.0.1

        smtp_connect_timeout 30

        router_id LVS1          # 设置lvs的id,在一个网络内应该是唯一的

}

vrrp_sync_group test {           #设置vrrp组

group {

        loadbalance

}

}

vrrp_instance loadbalance {

        state MASTER       #设置lvs的状态,报错MASTER和BACKUP两种,必须大写

        interface eth0     #设置对外服务的接口

        lvs_sync_daemon_inteface eth0   #设置lvs监听的接口

        virtual_router_id 51                     #设置虚拟路由表示

        priority 180            #设置优先级,数值越大,优先级越高

        advert_int 5           #设置同步时间间隔

authentication {                    #设置验证类型和密码

        auth_type PASS

        auth_pass 1111

}

virtual_ipaddress {                #设置lvs vip

        192.168.1.115

}

}

virtual_server 192.168.1.115 80 {

        delay_loop 6          #健康检查时间间隔

        lb_algo rr               #负载均衡调度算法

        lb_kind DR            #负载均衡转发规则

        #persistence_timeout 20  #设置会话保持时间,对bbs等很有用

        protocol TCP                #协议

        real_server 192.168.1.105 80 {

        weight 3                #设置权重

        TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

}

}

        real_server 192.168.1.103 80 {

        weight 3

        TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

}

}

        real_server 192.168.1.104 80 {

        weight 3

        TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

}

}

}

LVS-backup的配置文件如下

global_defs {

notification_email {

               jimo291@gmail.com

        }

        notification_email_from jimo291@gmail.com

        smtp_server 127.0.0.1

        smtp_connect_timeout 30

        router_id LVS2

}

vrrp_sync_group test {

group {

        loadbalance

}

}

vrrp_instance loadbalance {

        state BACKUP

        interface eth0

        lvs_sync_daemon_inteface eth0

        virtual_router_id 51

        priority 150

        advert_int 5

authentication {

        auth_type PASS

        auth_pass 1111

}

virtual_ipaddress {

        192.168.1.115

}

}

virtual_server 192.168.1.115 80 {

        delay_loop 6

        lb_algo rr

        lb_kind DR

        #persistence_timeout 20

        protocol TCP

        real_server 192.168.1.105 80 {

        weight 3

        TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

}

}

        real_server 192.168.1.103 80 {

        weight 3

        TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

}

}

        real_server 192.168.1.104 80 {

        weight 3

        TCP_CHECK {

        connect_timeout 3

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

}

}

}

      

6.         Realserver的配置

三台客户端的脚本都一样!

cat /etc/rc.d/init.d/realserver.sh

#!/bin/bash

# description: Config realserver lo and apply noarp

SNS_VIP=192.168.1.115

/etc/rc.d/init.d/functions

case "$1" in

start)

       ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP

       /sbin/route add -host $SNS_VIP dev lo:0

       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

       sysctl -p >/dev/null 2>&1

       echo "RealServer Start OK"

       ;;

stop)

       ifconfig lo:0 down

       route del $SNS_VIP >/dev/null 2>&1

       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

       echo "RealServer Stoped"

       ;;

*)

       echo "Usage: $0 {start|stop}"

       exit 1

esac

exit 0

7.         测试

1)        首先测试各个readserver,确定各个readserver都能正常访问,测试realserver.sh脚本,看启动后能不能绑定lvs vip,停止后能不能去除绑定的vip

2)        测试lvs能否进行负载均衡转发,利用ipvsadm命令查看,如果想只当当前的请求转发到哪个服务器去了,可以用ipvsadm加选项来查看,其完整形式为:ipvsadm –lcn | grep 192.168.1.115

3)        停掉主lvs看lvs backup是否接管!

4)        更详细信息请查看日志,tail -f /var/log/messages

本文由金沙棋牌发布于操作系统,转载请注明出处:keepalived负载均衡实战,LVS实现高可用集群

关键词:

上一篇:linux系统的休眠与唤醒简介,电源管理ACPI

下一篇:没有了