技術文章 » Nginx + Keepalived 高可用叢集部署

Nginx + Keepalived 高可用叢集部署

2023-03-09 12:01:20

負載均衡技術對於一個網站尤其是大型網站的web伺服器叢集來說是至關重要的！做好負載均衡架構，可以實現故障轉移和高可用環境，避免單點故障，保證網站健康持續執行。在使用 Nginx 做反向代理或者負載均衡的時候，都是以 Nginx 為入口，如果 Nginx 宕機了，那麼所有的服務都無法正常提供，影響非常嚴重。

為了避免負載均衡伺服器宕機造成嚴重影響，就需要建立一個備份機。主伺服器和備份機上都執行高可用（High Availability）監控程式，通過傳送諸如「I am alive」這樣的資訊來監控對方的執行狀況。當備份機不能在一定的時間內收到這樣的資訊時，它就接管主伺服器的服務IP並繼續提供負載均衡服務；當備份管理器又從主管理器收到「I am alive」這樣的資訊時，它就釋放服務IP地址，這樣的主伺服器就開始再次提供負載均衡服務。

高可用（High Availability）是分散式系統架構設計中必須考慮的因素之一，它通常是指，通過設計減少系統不能提供服務的時間。如果一個系統能夠一直提供服務，那麼這個可用性則是百分之百，但是我們不能保證一個系統能永遠不出問題，所以我們只能通過設計來儘可能的去減少由於系統的故障所造成的影響。

由於業務擴充套件，網站的存取量不斷加大，負載越來越高。現需要在web前端放置nginx負載均衡,同時結合keepalived對前端nginx實現HA高可用。

前文分享了《Linux下Nginx基礎應用》，《Linux下實現高可用軟體-Keepalived基礎知識梳理》；今天簡單分享Nginx + Keepalived 高可用集群部署。

主從叢集架構圖

環境說明
hostname ip 說明
Client-01 172.16.70.171 使用者端測試機
KeepMaster 172.16.70.181 keepalived 主伺服器 (nginx 主負載均衡器)
KeepBackup 172.16.70.182 keepalived 備伺服器 (nginx 備負載均衡器)
VIP 172.16.70.183 vrrp HA 虛擬地址，可有多個IP
Web1 172.16.70.191 後端web主伺服器1 (nginx 站點)
Web2 172.16.70.192 後端web備伺服器2 (nginx 站點)

準備環境

# 本次所有部署伺服器都設定
# cat /etc/redhat-release 
CentOS Linux release 7.9.2009 (Core)
# uname -r
3.10.0-1160.83.1.el7.x86_64

# systemctl stop firewalld
# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/sysconfig/selinux
# setenforce 0
# ntpdate 0.centos.pool.ntp.org
# yum install net-tools vim wget curl -y

 搭建後端web伺服器

  • 主備一樣操作
# 這裡以 web-01 為例
[root@web-01 ~]# wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~]# rpm -ivh nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~]# nginx -v
nginx version: nginx/1.20.2

[root@web-01 ~]# #echo "`hostname` `ifconfig ens33 |sed -n 's#.*inet \(.*\)netmask.*#\1#p'`" > /usr/share/nginx/html/index.html
[root@web-01 ~]# cat /usr/share/nginx/html/index.html
web-01 172.16.70.191 

[root@web-01 ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

[root@web-01 ~]# systemctl start nginx
[root@web-01 ~]# systemctl enable nginx
[root@web-01 ~]# netstat -ntupl | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      10687/nginx: master 
[root@web-01 ~]# ps -ef | grep nginx
root      10687      1  0 16:36 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     10688  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10689  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10690  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10691  10687  0 16:36 ?        00:00:00 nginx: worker process
root      10761  10586  0 16:45 pts/1    00:00:00 grep --color=auto nginx
 
# Client-01 測試存取
[root@Client-01 ~]# curl 172.16.70.191
web-01 172.16.70.191

 瀏覽器測試存取 http://ip/

 

Keep伺服器上部署nginx負載均衡器

  • 主備一樣操作
# 這裡以 KeepMaster 為例
# 安裝部署nginx
[root@KeepMaster ~]# wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~]# rpm -vih nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~]# nginx -v
nginx version: nginx/1.18.0

# 新建
[root@KeepMaster ~]# cat /etc/nginx/conf.d/web.conf
upstream web {
    server 172.16.70.191:80 weight=1 max_fails=3 fail_timeout=20s;
    server 172.16.70.192:80 weight=2 max_fails=3 fail_timeout=20s;
}
    # weight(權重)和存取比率成正比，預設值為1
    # max_fails 為允許失敗的次數，預設值為1
    # fail_timeout 當max_fails次失敗後，暫停將請求分發到該後端伺服器的時間

server {
  listen 80;
  server_name www.zhangwencheng.org;
  
  location / {
    proxy_pass http://web;
    proxy_set_header HOST $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
}

[root@KeepMaster ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

[root@KeepMaster ~]# systemctl start nginx
[root@KeepMaster ~]# systemctl enable nginx

[root@KeepMaster ~]# ps -ef | grep nginx
root       1677      1  0 17:28 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx      1678   1677  0 17:28 ?        00:00:00 nginx: worker process
root       1708   1444  0 17:33 pts/0    00:00:00 grep --color=auto nginx
[root@KeepMaster ~]# netstat -tnpl | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1677/nginx: master

測試機Client-01驗證負載均衡

  • Keep主備伺服器上的nginx負載均衡
# 在測試機上新增host解析, KeepMaster/KeepBackup主機IP
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.181	www.zhangwencheng.org
172.16.70.182	www.zhangwencheng.org

# 測試時候輪流關閉Keep節點，關閉後還是能夠存取並看到輪循效果即表示nginx負載均衡器叢集搭建成功。
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191

Keep伺服器上部署keepalived

  • 主備一樣操作
[root@KeepMaster ~]# yum install -y openssl openssl-devel libnl libnl-devel gcc
[root@KeepMaster ~]# mkdir /data/apps/keepalived -p
[root@KeepMaster ~]# wget --no-check-certificate  http://www.keepalived.org/software/keepalived-2.2.4.tar.gz
[root@KeepMaster ~]# tar -xf keepalived-2.2.4.tar.gz
[root@KeepMaster ~]# cd keepalived-2.2.4
[root@KeepMaster keepalived-2.2.4]# ls
aclocal.m4  autogen.sh   build-aux    ChangeLog  configure.ac  COPYING  INSTALL     keepalived.spec.in  m4           Makefile.in  snap  tools
AUTHOR      bin_install  build_setup  configure  CONTRIBUTORS  doc      keepalived  lib                 Makefile.am  README.md    TODO

[root@KeepMaster keepalived-2.2.4]# ./configure --prefix=/data/apps/keepalived
....
....
# 最後編譯正常輸出如下
Keepalived configuration
------------------------
Keepalived version       : 2.2.4
Compiler                 : gcc gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Preprocessor flags       : -D_GNU_SOURCE
Compiler flags           : -g -g -O2 -Wextra -Wunused -Wstrict-prototypes -Wabi -Wbad-function-cast -Wcast-align -Wcast-qual -Wdisabled-optimization -Wdouble-promotion \
-Wfloat-equal -Wframe-larger-than=5120 -Winit-self -Winline -Winvalid-pch -Wjump-misses-init -Wlogical-op -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs \
-Wmissing-prototypes -Wnested-externs -Wold-style-definition -Woverlength-strings -Wpointer-arith -Wredundant-decls -Wshadow -Wstack-protector -Wstrict-overflow=4 -Wsuggest-attribute=format \
-Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wsync-nand -Wtrampolines -Wundef -Wuninitialized -Wunknown-pragmas -Wunsafe-loop-optimizations -Wunsuffixed-float-constants -Wvariadic-macros \
-Wwrite-strings -fPIE -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -O2
Linker flags             : -pie -Wl,-z,relro -Wl,-z,now
Extra Lib                : -lm -lcrypto -lssl -lnl
Use IPVS Framework       : Yes
IPVS use libnl           : Yes
IPVS syncd attributes    : No
IPVS 64 bit stats        : No
HTTP_GET regex support   : No
fwmark socket support    : Yes
Use VRRP Framework       : Yes
Use VRRP VMAC            : Yes
Use VRRP authentication  : Yes
With track_process       : Yes
With linkbeat            : Yes
Use BFD Framework        : No
SNMP vrrp support        : No
SNMP checker support     : No
SNMP RFCv2 support       : No
SNMP RFCv3 support       : No
DBUS support             : No
Use JSON output          : No
libnl version            : 1
Use IPv4 devconf         : No
Use iptables             : No
Use nftables             : No
init type                : systemd
systemd notify           : No
Strict config checks     : No
Build documentation      : No
Default runtime options  : -D

[root@KeepMaster keepalived-2.2.4]# make -j 4 && make install
[root@KeepMaster keepalived-2.2.4]# ls
aclocal.m4  bin          build_setup  config.status  CONTRIBUTORS  INSTALL          keepalived.spec.in  Makefile     README     TODO
AUTHOR      bin_install  ChangeLog    configure      COPYING       keepalived       lib                 Makefile.am  README.md  tools
autogen.sh  build-aux    config.log   configure.ac   doc           keepalived.spec  m4                  Makefile.in  snap

[root@KeepMaster keepalived-2.2.4]# cp keepalived/keepalived /usr/local/sbin/ -a
[root@KeepMaster keepalived-2.2.4]# keepalived -v
Keepalived v2.2.4 (08/21,2021)

Copyright(C) 2001-2021 Alexandre Cassen, <[email protected]>

Built with kernel headers for Linux 3.10.0
Running on Linux 3.10.0-1160.83.1.el7.x86_64 #1 SMP Wed Jan 25 16:41:43 UTC 2023
Distro: CentOS Linux 7 (Core)

configure options: --prefix=/data/apps/keepalived

Config options:  LVS VRRP VRRP_AUTH VRRP_VMAC OLD_CHKSUM_COMPAT INIT=systemd

System options:  VSYSLOG LIBNL1 RTA_ENCAP RTA_EXPIRES RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTA_VIA IFA_FLAGS \
NET_LINUX_IF_H_COLLISION LIBIPTC_LINUX_NET_IF_H_COLLISION LIBIPVS_NETLINK IFLA_LINK_NETNSID GLOB_BRACE GLOB_ALTDIRFUNC INET6_ADDR_GEN_MODE SO_MARK

[root@KeepMaster keepalived-2.2.4]# cd /data/apps/keepalived/
[root@KeepMaster keepalived]# ls
bin  etc  sbin  share

[root@KeepMaster keepalived]# mv etc/keepalived/keepalived.conf etc/keepalived/keepalived.conf_bak
[root@KeepMaster keepalived]# vim etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_script chk_nginx {     
    script "/data/apps/keepalived/chk_nginx.sh"  
    interval 2                  
    weight -5                                   
}

vrrp_instance VI_1 {
    state MASTER　　　　# 備伺服器這為 BACKUP
    interface ens33
    virtual_router_id 51
    priority 110　　　　# 備伺服器這小於110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.183 
    }
track_script {
   chk_nginx
    }
}

優先順序不會不斷的提高或者降低，最終優先順序的範圍是在[1,254]，不會出現優先順序小於等於0或者優先順序大於等於255的情況。
在MASTER節點的vrrp_instance中設定nopreempt，當它異常恢復後，即使它prio更高也不會搶佔，這樣可以避免正常情況下做無謂的切換。

nginx檢測指令碼

編寫指令碼來判斷本機nginx是否正常，如果發現NginX不正常，自重啟nginx。等待2秒再次校驗，仍然失敗則不再嘗試，關閉keepalived，讓其他主機此時會接管VIP。
此指令碼必須在keepalived服務執行的前提下才有效！如果在keepalived服務先關閉的情況下，那麼nginx服務關閉後就不能實現自啟動了。

[root@KeepMaster keepalived]# cat chk_nginx.sh 
#!/bin/bash
chk=$(ps -C nginx --no-heading|wc -l)
if [ "${chk}" = "0" ]; then
    systemctl start nginx
    sleep 2
    chk=$(ps -C nginx --no-heading|wc -l)
    if [ "${chk}" = "0" ]; then
        systemctl stop keepalived
    fi
fi

[root@KeepMaster keepalived]# chmod +x chk_nginx.sh

測試機Client-01驗證VIP

  • Keep伺服器上的VIP
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.183	www.zhangwencheng.org

[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192

故障轉移測試

#手動關閉Master機器上的nginx服務，最多2秒鐘後就會自啟動
[root@KeepMaster ~]# systemctl stop nginx
[root@KeepMaster ~]# ps -ef | egrep 'nginx|keepalived'
root      57266      1  0 16:21 ?        00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      57267  57266  0 16:21 ?        00:00:01 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      60019      1  0 16:42 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     60020  60019  0 16:42 ?        00:00:00 nginx: worker process
root      60027   1444  0 16:42 pts/0    00:00:00 grep -E --color=auto nginx|keepalived
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.183/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

#手動關閉Master機器上的keepalived服務，已經發現沒VIP了
[root@KeepMaster ~]# systemctl stop keepalived
[root@KeepMaster ~]# ps -ef | egrep 'nginx|keepalived'
root      60019      1  0 16:42 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     60020  60019  0 16:42 ?        00:00:00 nginx: worker process
root      60348   1444  0 16:45 pts/0    00:00:00 grep -E --color=auto nginx|keepalived
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

[root@KeepMaster keepalived]# tail /var/log/messages
Mar  7 17:01:01 Keepalived-01 systemd: Started Session 27 of user root.
Mar  7 17:01:28 Keepalived-01 Keepalived[60703]: Stopping
Mar  7 17:01:28 Keepalived-01 systemd: Stopping LVS and VRRP High Availability Monitor...
Mar  7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) sent 0 priority
Mar  7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) removing VIPs.
Mar  7 17:01:28 Keepalived-01 NetworkManager[570]: <info>  [1678179688.0176] policy: set-hostname: current hostname was changed outside NetworkManager: 'KeepMaster'
Mar  7 17:01:29 Keepalived-01 Keepalived_vrrp[60704]: Stopped - used (self/children) 0.005506/0.797936 user time, 0.172766/0.818969 system time
Mar  7 17:01:29 Keepalived-01 Keepalived[60703]: CPU usage (self/children) user: 0.000000/0.803442 system: 0.001394/0.994146
Mar  7 17:01:29 Keepalived-01 Keepalived[60703]: Stopped Keepalived v2.2.4 (08/21,2021)
Mar  7 17:01:29 Keepalived-01 systemd: Stopped LVS and VRRP High Availability Monitor
  • KeepBackup檢視，已經接管VIP
[root@KeepBackup keepalived]# ps -ef | egrep 'nginx|keepalived'
root      65036      1  0 17:01 ?        00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      65037  65036  0 17:01 ?        00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      65067      1  0 17:01 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     65068  65067  0 17:01 ?        00:00:00 nginx: worker process
root      65122   1514  0 17:01 pts/0    00:00:00 grep -E --color=auto nginx|keepalived

[root@KeepBackup keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.182/24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.183/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
[root@KeepBackup keepalived]# tail /var/log/messages 
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 NetworkManager[573]: <info>  [1678179688.8137] policy: set-hostname: current hostname was changed outside NetworkManager: 'KeepBackup'
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: (VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
 
 
 

雙機高可用一般是通過虛擬IP（飄移IP）方法來實現的，基於Linux/Unix的IP別名技術，目前分為兩種：

  • 雙機主從模式：即前端使用兩臺伺服器，一臺主伺服器和一臺熱備伺服器，正常情況下，主伺服器繫結一個公網虛擬IP，提供負載均衡服務，熱備伺服器處於空閒狀態；當主伺服器發生故障時，熱備伺服器接管主伺服器的公網虛擬IP，提供負載均衡服務；但是熱備伺服器在主機器不出現故障的時候，永遠處於浪費狀態，對於伺服器不多的網站，該方案不經濟實惠。
  • 雙機主主模式：即前端使用兩臺負載均衡伺服器，互為主備，且都處於活動狀態，同時各自繫結一個公網虛擬IP，提供負載均衡服務；當其中一臺發生故障時，另一臺接管發生故障伺服器的公網虛擬IP（這時由非故障機器一臺負擔所有的請求）。這種方案，經濟實惠，非常適合於當前架構環境。

主主模式叢集架構圖

當了解主備模式後，雙主模式就容易設定多了。只需要在每臺keepalived組態檔，加上一個vrrp_instance命名vrrp_instance VI_2即可，更改幾個引數，設定另一個VIP：172.16.70.184

  • KeepMaster：state BACKUP ，priority 100， virtual_router_id 52
  • KeepBackup：state MASTER ，priority 110， virtual_router_id 52

KeepMaster上的keepalived.conf

[root@KeepMaster keepalived]# cat etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_script chk_nginx {     
    script "/data/apps/keepalived/chk_nginx.sh"  
    interval 2                  
    weight -5                   
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.183 
    }
track_script {
   chk_nginx
    }
}

vrrp_instance VI_2 {
    state BACKUP
    interface ens33
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.184
    }
track_script {
   chk_nginx
    }
}

[root@KeepMaster keepalived]# systemctl restart keepalived

 KeepBackup上的keepalived.conf

[root@KeepBackup keepalived]# cat etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_script chk_nginx {     
    script "/data/apps/keepalived/chk_nginx.sh"  
    interval 2                  
    weight -5                   
}

vrrp_instance VI_1 {
    state BAKCUP
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.183 
    }
track_script {
   chk_nginx
    }
}

vrrp_instance VI_2 {
    state MASTER
    interface ens33
    virtual_router_id 52
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.184
    }
track_script {
   chk_nginx
    }
}

[root@KeepBackup keepalived]# systemctl restart keepalived

 測試機Client-01驗證VIP2

  • Keep伺服器上的VIP2
# 註釋VIP1解析，此時僅測試VIP2；測試成功後再取消VIP1註釋
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
#172.16.70.183	www.zhangwencheng.org
172.16.70.184	www.zhangwencheng.org

[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191
  •  檢視VIP情況
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.183/32 scope global ens33　　　　# VIP1
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

[root@KeepBackup ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.182/24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.184/32 scope global ens33　　　　# VIP2
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
  •  故障轉移測試類同，不再贅述。