作業5

2020-10-20 01:00:21

1、如果主節點已經執行了一段時間,且有大量資料時,如何設定並啟動slave節點(寫出操作步驟)

  • 1、通過備份恢復資料至從伺服器

    1、在主伺服器完全備份
    mysqldump -A -F --single-transaction --master-data=1 >
    /backup/fullbackup.sql
    
    2、將備份的資料拷貝到新的從節點伺服器相關目錄下
    
  • 2、複製起始位置為備份時,二進位制紀錄檔檔案及其POS

    1、從節點伺服器進行相關設定
    注意:先檢視完全備份的二進位制檔案的節點位置,從完全備份的位置之後開始複製
    
    2、在從節點進行完全備份恢復
    
    3、從節點啟動slave服務
    

2、當master伺服器宕機,提升一個slave成為新的master(寫出操作步驟)

  • 1、找到哪個從節點的資料庫是最新,讓它成為新master

  • 2、新master修改組態檔,關閉read-only設定

    vim /etc/my.cnf.d/mariadb-server.cnf
    read-only=OFF
    
  • 3、清除舊的master複製資訊

    # 登入MySQL
    set global read_only=off;
    stop slave;
    reset slave all;
    
  • 4、在新master上完全備份

    • mysqldump -A --single-transaction --master-data=1 -F >
      backup.sql
      
    • 分析舊的master 的二進位制紀錄檔,將未同步到至新master的二進位制紀錄檔匯出來,恢復到新master,儘可能 恢復資料

  • 5、其它所有 slave 重新還原資料庫,指向新的master

    • 注意:同步恢復資料時,暫時停用二進位制檔案

      set sql_log_bin=off;
      

3、通過 MHA 0.58 搭建一個資料庫叢集結構

MySQL8.0-基於MHA的MySQL高可用架構搭建

在這裡插入圖片描述

(1)從宕機崩潰的master儲存二進位制紀錄檔事件(binlog events);

(2)識別含有最新更新的slave;

(3)應用差異的中繼紀錄檔(relay log)到其他的slave;

(4)應用從master儲存的二進位制紀錄檔事件(binlog events);

(5)提升一個slave為新的master;

(6)使其他的slave連線新的master進行復制;

相關命令

# 檢查ssh連線
[root@centos771 ~]# masterha_check_ssh --conf=/etc/masterha/app1.cnf

# 檢查MySQL複製狀況
[root@centos771 ~]# masterha_check_repl --conf=/etc/masterha/app1.cnf 

# 啟動MHA
[root@centos771 ~]# masterha_manager --conf=/etc/masterha/app1.cnf

# 檢測當前MHA執行狀態
[root@centos771 ~]# masterha_check_status --conf=/etc/masterha/app1.cnf 

# 啟動MHA
[root@centos771 ~]# masterha_manager --conf=/etc/masterha/app1.cnf

# 檢測master是否gangji
[root@centos771 ~]# masterha_master_monitor --conf=/etc/masterha/app1.cnf

# 手動切換master
[root@centos771 ~]# masterha_master_switch  --conf=/etc/masterha/app1.cnf --master_state=alive --new_master_host=10.0.0.80 --new_master_port=3306 --orig_master_is_new_slave --running_updates_limit=10000


注意:MHA目前不支援mariadb資料庫

實驗環境:虛擬機器器四臺

SQL檔案:hellodb.sql

  • master:centos80

    • ip:10.0.0.80

    • MySQL8.0

  • slave1(備用master):centos81

    • ip:10.0.0.81

    • MySQL8.0

  • slave2:centos82

    • ip:10.0.0.82

    • MySQL8.0

  • slave3(MHA):centos77

    • ip:10.0.0.77
    • MySQL8.0

前期準備

# 關閉防火牆,關閉selinux
[root@80 ~]# systemctl disable --now firewalld
[root@80 ~]# sed -ri '/^SELINUX=/s/(^SELINUX=)(.*)/\1disabled/' /etc/selinux/config

# 安裝MySQL8.0
[root@80 ~]# yum -y install mysql-server
[root@80 ~]# systemctl enable mysqld
[root@80 ~]# systemctl start mysqld

1 首先設定好,基於GTID的主從半同步複製(一主兩從)

master

[root@hah ~]# vim /etc/my.cnf.d/mysql-server.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysql/mysqld.log
pid-file=/run/mysqld/mysqld.pid

server_id=80
gtid_mode=ON
enforce_gtid_consistency=ON
log_bin
log_slave_updates=ON

[root@hah ~]# systemctl restart mysqld


slave1

[root@81 ~]# vim /etc/my.cnf.d/mysql-server.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysql/mysqld.log
pid-file=/run/mysqld/mysqld.pid

server_id=81
gtid_mode=ON
enforce_gtid_consistency=ON
log_bin
log_slave_updates=ON

[root@81 ~]# systemctl restart mysqld

slave2

[root@82 ~]# vim /etc/my.cnf.d/mysql-server.cnf

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysql/mysqld.log
pid-file=/run/mysqld/mysqld.pid

server_id=82
gtid_mode=ON
enforce_gtid_consistency=ON
log_bin
log_slave_updates=ON

[root@82 ~]# systemctl restart mysqld

master

# 建立複製使用者並授權
mysql> create user centos@'10.0.0.%' identified by '123456';
Query OK, 0 rows affected (0.17 sec)

mysql> grant replication slave on *.* to 'centos'@'10.0.0.%';
Query OK, 0 rows affected (0.06 sec)

# 安裝外掛
mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so';
Query OK, 0 rows affected (0.04 sec)

mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
Query OK, 0 rows affected (0.06 sec)

mysql> set global rpl_semi_sync_master_enabled=1;
Query OK, 0 rows affected (0.00 sec)

mysql> set global rpl_semi_sync_master_timeout=3000;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like '%rpl%';
+-------------------------------------------+------------+
| Variable_name                             | Value      |
+-------------------------------------------+------------+
| rpl_read_size                             | 8192       |
| rpl_semi_sync_master_enabled              | ON         |
| rpl_semi_sync_master_timeout              | 3000       |
| rpl_semi_sync_master_trace_level          | 32         |
| rpl_semi_sync_master_wait_for_slave_count | 1          |
| rpl_semi_sync_master_wait_no_slave        | ON         |
| rpl_semi_sync_master_wait_point           | AFTER_SYNC |
| rpl_semi_sync_slave_enabled               | OFF        |
| rpl_semi_sync_slave_trace_level           | 32         |
| rpl_stop_slave_timeout                    | 31536000   |
+-------------------------------------------+------------+
10 rows in set (0.01 sec)

slave1

# 複製執行緒
mysql> change master to master_host='10.0.0.80',master_user='centos',master_password='123456',master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.04 sec)

# 安裝外掛
mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so';
Query OK, 0 rows affected (0.08 sec)

mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
Query OK, 0 rows affected (0.10 sec)

mysql> set global rpl_semi_sync_slave_enabled=1;
Query OK, 0 rows affected (0.00 sec)

# 啟動從執行緒
mysql> start slave;
Query OK, 0 rows affected (0.10 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.0.80
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: hah-bin.000003
          Read_Master_Log_Pos: 672
               Relay_Log_File: 81-relay-bin.000002
                Relay_Log_Pos: 882
        Relay_Master_Log_File: hah-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 672
              Relay_Log_Space: 1087
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 80
                  Master_UUID: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-2
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

slave2

mysql> change master to master_host='10.0.0.80',master_user='centos',master_password='123456',master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.05 sec)

mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so';
Query OK, 0 rows affected (0.01 sec)

mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
Query OK, 0 rows affected (0.10 sec)

mysql> set global rpl_semi_sync_slave_enabled=1;
Query OK, 0 rows affected (0.00 sec)

mysql> start slave;
Query OK, 0 rows affected (0.05 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.0.80
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: hah-bin.000003
          Read_Master_Log_Pos: 672
               Relay_Log_File: 82-relay-bin.000002
                Relay_Log_Pos: 882
        Relay_Master_Log_File: hah-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 672
              Relay_Log_Space: 1087
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 80
                  Master_UUID: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-2
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

測試主從複製

# master
mysql> create database student;
Query OK, 1 row affected (0.02 sec)

# slave1
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| student            |
| sys                |
+--------------------+
5 rows in set (0.01 sec)

# slave2
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| student            |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

2 設定ssh免密登入

1、生成ssh登入金鑰

# MAH
[root@centos771 ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:aEKIhERoMc+c7LhcmHHfz04TtkXZ5mvexBq7y/g2QPE root@83
The key's randomart image is:
+---[RSA 2048]----+
|=*.              |
|+o*..      .o    |
|o..B.      ooo   |
|  B.. ..  ..oE   |
| + o..o.So.. .   |
|. o  o  + +.  o  |
| o       *  .+ o |
|        o . =o*  |
|         . .oO+. |
+----[SHA256]-----+
[root@centos771 ~]# ssh-copy-id 10.0.0.71


# 生成公鑰和私鑰傳送到server1,2,3上,使之互相之間可以進行免密登入
[root@centos771 ~]# scp -r .ssh 10.0.0.80:/root/
[root@centos771 ~]# scp -r .ssh 10.0.0.81:/root/
[root@centos771 ~]# scp -r .ssh 10.0.0.82:/root/

3 設定MHA

相關下載連結

  • https://github.com/yoshinorim/mha4mysql-node/releases/download/v0.58/mha4mysql-node-0.58-0.el7.centos.noarch.rpm
  • https://github.com/yoshinorim/mha4mysql-manager/releases/download/v0.58/mha4mysql-manager-0.58-0.el7.centos.noarch.rpm
  • https://github.com/yoshinorim/mha4mysql-manager/releases/download/v0.58/mha4mysql-manager-0.58.tar.gz
[root@centos771 ~]# yum -y install mha4mysql-node-0.58-0.el7.centos.noarch.rpm
[root@centos771 ~]# yum -y install mha4mysql-manager-0.58-0.el7.centos.noarch.rpm 

# 將mha4mysql-node傳送到另外三臺主機
[root@centos771 ~]# rsync /data/mha4mysql-node-0.58-0.el7.centos.noarch.rpm 10.0.0.80:/root/
[root@centos771 ~]# rsync /data/mha4mysql-node-0.58-0.el7.centos.noarch.rpm 10.0.0.81:/root/
[root@centos771 ~]# rsync /data/mha4mysql-node-0.58-0.el7.centos.noarch.rpm 10.0.0.82:/root/

# master,slave1,slave2 安裝node

編寫組態檔

[root@centos771 ~]# mkdir /etc/masterha/
[root@centos771 ~]# vim /etc/masterha/app.cnf
[server default]
manager_workdir=/etc/masterha
manager_log=/var/log/masterha.log
master_binlog_dir=/var/lib/mysql
user=root
password=123456
ping_interval=1
remote_workdir=/tmp
repl_password=123456
repl_user=centos
ssh_user=root

[server1]
hostname=10.0.0.80
port=3306

[server2]
hostname=10.0.0.81
port=3306
candidate_master=1
check_repl_delay=0

[server3]
hostname=10.0.0.82
port=3306
no_master=1

測試連線

[root@centos771 ~]# masterha_check_ssh --conf=/etc/masterha/app.cnf
Thu Oct 15 17:17:59 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Oct 15 17:17:59 2020 - [info] Reading application default configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:17:59 2020 - [info] Reading server configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:17:59 2020 - [info] Starting SSH connection tests..
Thu Oct 15 17:18:00 2020 - [debug] 
Thu Oct 15 17:17:59 2020 - [debug]  Connecting via SSH from root@10.0.0.80(10.0.0.80:22) to root@10.0.0.81(10.0.0.81:22)..
Thu Oct 15 17:17:59 2020 - [debug]   ok.
Thu Oct 15 17:17:59 2020 - [debug]  Connecting via SSH from root@10.0.0.80(10.0.0.80:22) to root@10.0.0.82(10.0.0.82:22)..
Thu Oct 15 17:18:00 2020 - [debug]   ok.
Thu Oct 15 17:18:01 2020 - [debug] 
Thu Oct 15 17:17:59 2020 - [debug]  Connecting via SSH from root@10.0.0.81(10.0.0.81:22) to root@10.0.0.80(10.0.0.80:22)..
Thu Oct 15 17:18:00 2020 - [debug]   ok.
Thu Oct 15 17:18:00 2020 - [debug]  Connecting via SSH from root@10.0.0.81(10.0.0.81:22) to root@10.0.0.82(10.0.0.82:22)..
Thu Oct 15 17:18:01 2020 - [debug]   ok.
Thu Oct 15 17:18:01 2020 - [debug] 
Thu Oct 15 17:18:00 2020 - [debug]  Connecting via SSH from root@10.0.0.82(10.0.0.82:22) to root@10.0.0.80(10.0.0.80:22)..
Thu Oct 15 17:18:00 2020 - [debug]   ok.
Thu Oct 15 17:18:00 2020 - [debug]  Connecting via SSH from root@10.0.0.82(10.0.0.82:22) to root@10.0.0.81(10.0.0.81:22)..
Thu Oct 15 17:18:01 2020 - [debug]   ok.
Thu Oct 15 17:18:01 2020 - [info] All SSH connection tests passed successfully.

master

# 新增授權
mysql> create user root@'10.0.0.%' identified by '123456';
Query OK, 0 rows affected (0.02 sec)

mysql> grant all on *.* to 'root'@'10.0.0.%';
Query OK, 0 rows affected (0.05 sec)

slave

#slave1
mysql> set global read_only=1;
Query OK, 0 rows affected (0.01 sec)

#slave2
mysql> set global read_only=1;
Query OK, 0 rows affected (0.01 sec)

檢查整個複製環境狀況(健康檢查)

[root@centos771 ~]# masterha_check_repl --conf=/etc/masterha/app.cnf
Thu Oct 15 17:27:13 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Oct 15 17:27:13 2020 - [info] Reading application default configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:27:13 2020 - [info] Reading server configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:27:13 2020 - [info] MHA::MasterMonitor version 0.58.
Thu Oct 15 17:27:18 2020 - [info] GTID failover mode = 1
Thu Oct 15 17:27:18 2020 - [info] Dead Servers:
Thu Oct 15 17:27:18 2020 - [info] Alive Servers:
Thu Oct 15 17:27:18 2020 - [info]   10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:27:18 2020 - [info]   10.0.0.81(10.0.0.81:3306)
Thu Oct 15 17:27:18 2020 - [info]   10.0.0.82(10.0.0.82:3306)
Thu Oct 15 17:27:18 2020 - [info] Alive Slaves:
Thu Oct 15 17:27:18 2020 - [info]   10.0.0.81(10.0.0.81:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:27:18 2020 - [info]     GTID ON
Thu Oct 15 17:27:18 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:27:18 2020 - [info]     Primary candidate for the new Master (candidate_master is set)
Thu Oct 15 17:27:18 2020 - [info]   10.0.0.82(10.0.0.82:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:27:18 2020 - [info]     GTID ON
Thu Oct 15 17:27:18 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:27:18 2020 - [info]     Not candidate for the new Master (no_master is set)
Thu Oct 15 17:27:18 2020 - [info] Current Alive Master: 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:27:18 2020 - [info] Checking slave configurations..
Thu Oct 15 17:27:18 2020 - [info] Checking replication filtering settings..
Thu Oct 15 17:27:18 2020 - [info]  binlog_do_db= , binlog_ignore_db= 
Thu Oct 15 17:27:18 2020 - [info]  Replication filtering check ok.
Thu Oct 15 17:27:18 2020 - [info] GTID (with auto-pos) is supported. Skipping all SSH and Node package checking.
Thu Oct 15 17:27:18 2020 - [info] Checking SSH publickey authentication settings on the current master..
Thu Oct 15 17:27:19 2020 - [info] HealthCheck: SSH to 10.0.0.80 is reachable.
Thu Oct 15 17:27:19 2020 - [info] 
10.0.0.80(10.0.0.80:3306) (current master)
 +--10.0.0.81(10.0.0.81:3306)
 +--10.0.0.82(10.0.0.82:3306)

Thu Oct 15 17:27:19 2020 - [info] Checking replication health on 10.0.0.81..
Thu Oct 15 17:27:19 2020 - [info]  ok.
Thu Oct 15 17:27:19 2020 - [info] Checking replication health on 10.0.0.82..
Thu Oct 15 17:27:19 2020 - [info]  ok.
Thu Oct 15 17:27:19 2020 - [warning] master_ip_failover_script is not defined.
Thu Oct 15 17:27:19 2020 - [warning] shutdown_script is not defined.
Thu Oct 15 17:27:19 2020 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

測試

手動同步

(1)手動關閉server1(master)

[root@hah ~]# systemctl stop mysqld

(2)在server4(mha)上將master從server1手動同步到server1上

[root@centos771 ~]# masterha_master_switch --master_state=dead --conf=/etc/masterha/app.cnf --dead_master_ip=10.0.0.80 --dead_master_host=10.0.0.80 --dead_master_port=3306 --new_master_host=10.0.0.81 --new_master_port=3306

Thu Oct 15 17:31:29 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Oct 15 17:31:29 2020 - [info] Reading application default configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:31:29 2020 - [info] Reading server configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:31:29 2020 - [info] MHA::MasterFailover version 0.58.
Thu Oct 15 17:31:29 2020 - [info] Starting master failover.
Thu Oct 15 17:31:29 2020 - [info] 
Thu Oct 15 17:31:29 2020 - [info] * Phase 1: Configuration Check Phase..
Thu Oct 15 17:31:29 2020 - [info] 
Thu Oct 15 17:31:30 2020 - [info] GTID failover mode = 1
Thu Oct 15 17:31:30 2020 - [info] Dead Servers:
Thu Oct 15 17:31:30 2020 - [info]   10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:31:30 2020 - [info] Checking master reachability via MySQL(double check)...
Thu Oct 15 17:31:30 2020 - [info]  ok.
Thu Oct 15 17:31:30 2020 - [info] Alive Servers:
Thu Oct 15 17:31:30 2020 - [info]   10.0.0.81(10.0.0.81:3306)
Thu Oct 15 17:31:30 2020 - [info]   10.0.0.82(10.0.0.82:3306)
Thu Oct 15 17:31:30 2020 - [info] Alive Slaves:
Thu Oct 15 17:31:30 2020 - [info]   10.0.0.81(10.0.0.81:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:31:30 2020 - [info]     GTID ON
Thu Oct 15 17:31:30 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:31:30 2020 - [info]     Primary candidate for the new Master (candidate_master is set)
Thu Oct 15 17:31:30 2020 - [info]   10.0.0.82(10.0.0.82:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:31:30 2020 - [info]     GTID ON
Thu Oct 15 17:31:30 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:31:30 2020 - [info]     Not candidate for the new Master (no_master is set)
Master 10.0.0.80(10.0.0.80:3306) is dead. Proceed? (yes/NO): yes
Thu Oct 15 17:31:34 2020 - [info] Starting GTID based failover.
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] ** Phase 1: Configuration Check Phase completed.
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] * Phase 2: Dead Master Shutdown Phase..
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] HealthCheck: SSH to 10.0.0.80 is reachable.
Thu Oct 15 17:31:34 2020 - [error][/usr/share/perl5/vendor_perl/MHA/ManagerUtil.pm, ln122] Got error when getting node version. Error:
Thu Oct 15 17:31:34 2020 - [error][/usr/share/perl5/vendor_perl/MHA/ManagerUtil.pm, ln123] 
bash: apply_diff_relay_logs: command not found
Thu Oct 15 17:31:34 2020 - [warning] Failed to get MHA Node version from dead master. Guessing that SSH is NOT reachable.
Thu Oct 15 17:31:34 2020 - [info] Forcing shutdown so that applications never connect to the current master..
Thu Oct 15 17:31:34 2020 - [warning] master_ip_failover_script is not set. Skipping invalidating dead master IP address.
Thu Oct 15 17:31:34 2020 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.
Thu Oct 15 17:31:34 2020 - [info] * Phase 2: Dead Master Shutdown Phase completed.
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] * Phase 3: Master Recovery Phase..
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] * Phase 3.1: Getting Latest Slaves Phase..
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] The latest binary log file/position on all slaves is hah-bin.000003:1372
Thu Oct 15 17:31:34 2020 - [info] Retrieved Gtid Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
Thu Oct 15 17:31:34 2020 - [info] Latest slaves (Slaves that received relay log files to the latest):
Thu Oct 15 17:31:34 2020 - [info]   10.0.0.81(10.0.0.81:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:31:34 2020 - [info]     GTID ON
Thu Oct 15 17:31:34 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:31:34 2020 - [info]     Primary candidate for the new Master (candidate_master is set)
Thu Oct 15 17:31:34 2020 - [info]   10.0.0.82(10.0.0.82:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:31:34 2020 - [info]     GTID ON
Thu Oct 15 17:31:34 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:31:34 2020 - [info]     Not candidate for the new Master (no_master is set)
Thu Oct 15 17:31:34 2020 - [info] The oldest binary log file/position on all slaves is hah-bin.000003:1372
Thu Oct 15 17:31:34 2020 - [info] Retrieved Gtid Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
Thu Oct 15 17:31:34 2020 - [info] Oldest slaves:
Thu Oct 15 17:31:34 2020 - [info]   10.0.0.81(10.0.0.81:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:31:34 2020 - [info]     GTID ON
Thu Oct 15 17:31:34 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:31:34 2020 - [info]     Primary candidate for the new Master (candidate_master is set)
Thu Oct 15 17:31:34 2020 - [info]   10.0.0.82(10.0.0.82:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:31:34 2020 - [info]     GTID ON
Thu Oct 15 17:31:34 2020 - [info]     Replicating from 10.0.0.80(10.0.0.80:3306)
Thu Oct 15 17:31:34 2020 - [info]     Not candidate for the new Master (no_master is set)
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] * Phase 3.3: Determining New Master Phase..
Thu Oct 15 17:31:34 2020 - [info] 
Thu Oct 15 17:31:34 2020 - [info] 10.0.0.81 can be new master.
Thu Oct 15 17:31:34 2020 - [info] New master is 10.0.0.81(10.0.0.81:3306)
Thu Oct 15 17:31:34 2020 - [info] Starting master failover..
Thu Oct 15 17:31:34 2020 - [info] 
From:
10.0.0.80(10.0.0.80:3306) (current master)
 +--10.0.0.81(10.0.0.81:3306)
 +--10.0.0.82(10.0.0.82:3306)

To:
10.0.0.81(10.0.0.81:3306) (new master)
 +--10.0.0.82(10.0.0.82:3306)

Starting master switch from 10.0.0.80(10.0.0.80:3306) to 10.0.0.81(10.0.0.81:3306)? (yes/NO): yes
Thu Oct 15 17:31:41 2020 - [info] New master decided manually is 10.0.0.81(10.0.0.81:3306)
Thu Oct 15 17:31:41 2020 - [info] 
Thu Oct 15 17:31:41 2020 - [info] * Phase 3.3: New Master Recovery Phase..
Thu Oct 15 17:31:41 2020 - [info] 
Thu Oct 15 17:31:41 2020 - [info]  Waiting all logs to be applied.. 
Thu Oct 15 17:31:41 2020 - [info]   done.
Thu Oct 15 17:31:41 2020 - [info] Getting new master's binlog name and position..
Thu Oct 15 17:31:41 2020 - [info]  81-bin.000002:1407
Thu Oct 15 17:31:41 2020 - [info]  All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.0.0.81', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='centos', MASTER_PASSWORD='xxx';
Thu Oct 15 17:31:41 2020 - [info] Master Recovery succeeded. File:Pos:Exec_Gtid_Set: 81-bin.000002, 1407, 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
Thu Oct 15 17:31:41 2020 - [warning] master_ip_failover_script is not set. Skipping taking over new master IP address.
Thu Oct 15 17:31:41 2020 - [info] Setting read_only=0 on 10.0.0.81(10.0.0.81:3306)..
Thu Oct 15 17:31:41 2020 - [info]  ok.
Thu Oct 15 17:31:41 2020 - [info] ** Finished master recovery successfully.
Thu Oct 15 17:31:41 2020 - [info] * Phase 3: Master Recovery Phase completed.
Thu Oct 15 17:31:41 2020 - [info] 
Thu Oct 15 17:31:41 2020 - [info] * Phase 4: Slaves Recovery Phase..
Thu Oct 15 17:31:41 2020 - [info] 
Thu Oct 15 17:31:41 2020 - [info] 
Thu Oct 15 17:31:41 2020 - [info] * Phase 4.1: Starting Slaves in parallel..
Thu Oct 15 17:31:41 2020 - [info] 
Thu Oct 15 17:31:41 2020 - [info] -- Slave recovery on host 10.0.0.82(10.0.0.82:3306) started, pid: 17318. Check tmp log /etc/masterha/10.0.0.82_3306_20201015173129.log if it takes time..
Thu Oct 15 17:31:43 2020 - [info] 
Thu Oct 15 17:31:43 2020 - [info] Log messages from 10.0.0.82 ...
Thu Oct 15 17:31:43 2020 - [info] 
Thu Oct 15 17:31:41 2020 - [info]  Resetting slave 10.0.0.82(10.0.0.82:3306) and starting replication from the new master 10.0.0.81(10.0.0.81:3306)..
Thu Oct 15 17:31:42 2020 - [info]  Executed CHANGE MASTER.
Thu Oct 15 17:31:43 2020 - [info]  Slave started.
Thu Oct 15 17:31:43 2020 - [info]  gtid_wait(2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5) completed on 10.0.0.82(10.0.0.82:3306). Executed 0 events.
Thu Oct 15 17:31:43 2020 - [info] End of log messages from 10.0.0.82.
Thu Oct 15 17:31:43 2020 - [info] -- Slave on host 10.0.0.82(10.0.0.82:3306) started.
Thu Oct 15 17:31:43 2020 - [info] All new slave servers recovered successfully.
Thu Oct 15 17:31:43 2020 - [info] 
Thu Oct 15 17:31:43 2020 - [info] * Phase 5: New master cleanup phase..
Thu Oct 15 17:31:43 2020 - [info] 
Thu Oct 15 17:31:43 2020 - [info] Resetting slave info on the new master..
Thu Oct 15 17:31:44 2020 - [info]  10.0.0.81: Resetting slave info succeeded.
Thu Oct 15 17:31:44 2020 - [info] Master failover to 10.0.0.81(10.0.0.81:3306) completed successfully.
Thu Oct 15 17:31:44 2020 - [info] 

----- Failover Report -----

app: MySQL Master failover 10.0.0.80(10.0.0.80:3306) to 10.0.0.81(10.0.0.81:3306) succeeded

Master 10.0.0.80(10.0.0.80:3306) is down!

Check MHA Manager logs at centos771 for details.

Started manual(interactive) failover.
Selected 10.0.0.81(10.0.0.81:3306) as a new master.
10.0.0.81(10.0.0.81:3306): OK: Applying all logs succeeded.
10.0.0.82(10.0.0.82:3306): OK: Slave started, replicating from 10.0.0.81(10.0.0.81:3306)
10.0.0.81(10.0.0.81:3306): Resetting slave info succeeded.
Master failover to 10.0.0.81(10.0.0.81:3306) completed successfully.

(3)測試在server1上檢視slave狀態為空,server2上的master顯示是server2,說明手動轉換成功
server2:

mysql> show slave status\G
Empty set (0.00 sec)

server3:

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.0.81
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: 81-bin.000002
          Read_Master_Log_Pos: 1407
               Relay_Log_File: 82-relay-bin.000002
                Relay_Log_Pos: 407
        Relay_Master_Log_File: 81-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1407
              Relay_Log_Space: 612
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 81
                  Master_UUID: 711d79aa-0eb8-11eb-b3dc-000c299dcf4d
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

(4)然後開啟server1的mysqld,在sever1上重新新增master,檢視slave的狀態,顯示master是server2,切換成功

[root@hah ~]# systemctl start mysqld 
mysql> change master to master_host='10.0.0.81',master_port=3306,master_auto_position=1,master_user='centos',master_password='123456';
Query OK, 0 rows affected, 2 warnings (0.20 sec)

mysql> start slave;
Query OK, 0 rows affected (0.03 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.0.81
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: 81-bin.000002
          Read_Master_Log_Pos: 1407
               Relay_Log_File: hah-relay-bin.000002
                Relay_Log_Pos: 407
        Relay_Master_Log_File: 81-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1407
              Relay_Log_Space: 613
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 81
                  Master_UUID: 711d79aa-0eb8-11eb-b3dc-000c299dcf4d
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

兩個主備master都開啟

(1)首先刪除server4的app1.failover.complete,否則再次轉換不能成功

[root@centos771 ~]# ll /etc/masterha/
total 4
-rw-r--r--. 1 root root 444 Oct 15 17:00 app.cnf
-rw-r--r--. 1 root root   0 Oct 15 17:31 app.failover.complete
[root@centos771 ~]# rm -rf /etc/masterha/app.failover.complete   

(2)手動切換新的master——>server1

[root@centos771 ~]# masterha_master_switch  --conf=/etc/masterha/app.cnf --master_state=alive --new_master_host=10.0.0.80 --new_master_port=3306 --orig_master_is_new_slave --running_updates_limit=10000

Thu Oct 15 17:40:52 2020 - [info] MHA::MasterRotate version 0.58.
Thu Oct 15 17:40:52 2020 - [info] Starting online master switch..
Thu Oct 15 17:40:52 2020 - [info] 
Thu Oct 15 17:40:52 2020 - [info] * Phase 1: Configuration Check Phase..
Thu Oct 15 17:40:52 2020 - [info] 
Thu Oct 15 17:40:52 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Oct 15 17:40:52 2020 - [info] Reading application default configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:40:52 2020 - [info] Reading server configuration from /etc/masterha/app.cnf..
Thu Oct 15 17:40:53 2020 - [info] GTID failover mode = 1
Thu Oct 15 17:40:53 2020 - [info] Current Alive Master: 10.0.0.81(10.0.0.81:3306)
Thu Oct 15 17:40:53 2020 - [info] Alive Slaves:
Thu Oct 15 17:40:53 2020 - [info]   10.0.0.80(10.0.0.80:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:40:53 2020 - [info]     GTID ON
Thu Oct 15 17:40:53 2020 - [info]     Replicating from 10.0.0.81(10.0.0.81:3306)
Thu Oct 15 17:40:53 2020 - [info]   10.0.0.82(10.0.0.82:3306)  Version=8.0.13 (oldest major version between slaves) log-bin:enabled
Thu Oct 15 17:40:53 2020 - [info]     GTID ON
Thu Oct 15 17:40:53 2020 - [info]     Replicating from 10.0.0.81(10.0.0.81:3306)
Thu Oct 15 17:40:53 2020 - [info]     Not candidate for the new Master (no_master is set)

It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on 10.0.0.81(10.0.0.81:3306)? (YES/no): yes
Thu Oct 15 17:40:56 2020 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
Thu Oct 15 17:40:56 2020 - [info]  ok.
Thu Oct 15 17:40:56 2020 - [info] Checking MHA is not monitoring or doing failover..
Thu Oct 15 17:40:56 2020 - [info] Checking replication health on 10.0.0.80..
Thu Oct 15 17:40:56 2020 - [info]  ok.
Thu Oct 15 17:40:56 2020 - [info] Checking replication health on 10.0.0.82..
Thu Oct 15 17:40:56 2020 - [info]  ok.
Thu Oct 15 17:40:56 2020 - [info] 10.0.0.80 can be new master.
Thu Oct 15 17:40:56 2020 - [info] 
From:
10.0.0.81(10.0.0.81:3306) (current master)
 +--10.0.0.80(10.0.0.80:3306)
 +--10.0.0.82(10.0.0.82:3306)

To:
10.0.0.80(10.0.0.80:3306) (new master)
 +--10.0.0.82(10.0.0.82:3306)
 +--10.0.0.81(10.0.0.81:3306)

Starting master switch from 10.0.0.81(10.0.0.81:3306) to 10.0.0.80(10.0.0.80:3306)? (yes/NO): yes
Thu Oct 15 17:40:59 2020 - [info] Checking whether 10.0.0.80(10.0.0.80:3306) is ok for the new master..
Thu Oct 15 17:40:59 2020 - [info]  ok.
Thu Oct 15 17:40:59 2020 - [info] 10.0.0.81(10.0.0.81:3306): SHOW SLAVE STATUS returned empty result. To check replication filtering rules, temporarily executing CHANGE MASTER to a dummy host.
Thu Oct 15 17:40:59 2020 - [info] 10.0.0.81(10.0.0.81:3306): Resetting slave pointing to the dummy host.
Thu Oct 15 17:40:59 2020 - [info] ** Phase 1: Configuration Check Phase completed.
Thu Oct 15 17:40:59 2020 - [info] 
Thu Oct 15 17:40:59 2020 - [info] * Phase 2: Rejecting updates Phase..
Thu Oct 15 17:40:59 2020 - [info] 
master_ip_online_change_script is not defined. If you do not disable writes on the current master manually, applications keep writing on the current master. Is it ok to proceed? (yes/NO): yes
Thu Oct 15 17:41:02 2020 - [info] Locking all tables on the orig master to reject updates from everybody (including root):
Thu Oct 15 17:41:02 2020 - [info] Executing FLUSH TABLES WITH READ LOCK..
Thu Oct 15 17:41:02 2020 - [info]  ok.
Thu Oct 15 17:41:02 2020 - [info] Orig master binlog:pos is 81-bin.000002:1407.
Thu Oct 15 17:41:02 2020 - [info]  Waiting to execute all relay logs on 10.0.0.80(10.0.0.80:3306)..
Thu Oct 15 17:41:02 2020 - [info]  master_pos_wait(81-bin.000002:1407) completed on 10.0.0.80(10.0.0.80:3306). Executed 0 events.
Thu Oct 15 17:41:02 2020 - [info]   done.
Thu Oct 15 17:41:02 2020 - [info] Getting new master's binlog name and position..
Thu Oct 15 17:41:02 2020 - [info]  hah-bin.000004:195
Thu Oct 15 17:41:02 2020 - [info]  All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.0.0.80', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='centos', MASTER_PASSWORD='xxx';
Thu Oct 15 17:41:02 2020 - [info] 
Thu Oct 15 17:41:02 2020 - [info] * Switching slaves in parallel..
Thu Oct 15 17:41:02 2020 - [info] 
Thu Oct 15 17:41:02 2020 - [info] -- Slave switch on host 10.0.0.82(10.0.0.82:3306) started, pid: 17333
Thu Oct 15 17:41:02 2020 - [info] 
Thu Oct 15 17:41:04 2020 - [info] Log messages from 10.0.0.82 ...
Thu Oct 15 17:41:04 2020 - [info] 
Thu Oct 15 17:41:02 2020 - [info]  Waiting to execute all relay logs on 10.0.0.82(10.0.0.82:3306)..
Thu Oct 15 17:41:02 2020 - [info]  master_pos_wait(81-bin.000002:1407) completed on 10.0.0.82(10.0.0.82:3306). Executed 0 events.
Thu Oct 15 17:41:02 2020 - [info]   done.
Thu Oct 15 17:41:02 2020 - [info]  Resetting slave 10.0.0.82(10.0.0.82:3306) and starting replication from the new master 10.0.0.80(10.0.0.80:3306)..
Thu Oct 15 17:41:02 2020 - [info]  Executed CHANGE MASTER.
Thu Oct 15 17:41:03 2020 - [info]  Slave started.
Thu Oct 15 17:41:04 2020 - [info] End of log messages from 10.0.0.82 ...
Thu Oct 15 17:41:04 2020 - [info] 
Thu Oct 15 17:41:04 2020 - [info] -- Slave switch on host 10.0.0.82(10.0.0.82:3306) succeeded.
Thu Oct 15 17:41:04 2020 - [info] Unlocking all tables on the orig master:
Thu Oct 15 17:41:04 2020 - [info] Executing UNLOCK TABLES..
Thu Oct 15 17:41:04 2020 - [info]  ok.
Thu Oct 15 17:41:04 2020 - [info] Starting orig master as a new slave..
Thu Oct 15 17:41:04 2020 - [info]  Resetting slave 10.0.0.81(10.0.0.81:3306) and starting replication from the new master 10.0.0.80(10.0.0.80:3306)..
Thu Oct 15 17:41:04 2020 - [info]  Executed CHANGE MASTER.
Thu Oct 15 17:41:05 2020 - [info]  Slave started.
Thu Oct 15 17:41:05 2020 - [info] All new slave servers switched successfully.
Thu Oct 15 17:41:05 2020 - [info] 
Thu Oct 15 17:41:05 2020 - [info] * Phase 5: New master cleanup phase..
Thu Oct 15 17:41:05 2020 - [info] 
Thu Oct 15 17:41:05 2020 - [info]  10.0.0.80: Resetting slave info succeeded.
Thu Oct 15 17:41:05 2020 - [info] Switching master to 10.0.0.80(10.0.0.80:3306) completed successfully.

(3)master測試結果顯示如下(這次server2不用重新新增master)

mysql> show slave status\G
Empty set (0.00 sec)

server1:

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.0.80
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: hah-bin.000004
          Read_Master_Log_Pos: 195
               Relay_Log_File: 81-relay-bin.000002
                Relay_Log_Pos: 365
        Relay_Master_Log_File: hah-bin.000004
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 195
              Relay_Log_Space: 570
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 80
                  Master_UUID: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

server2

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.0.80
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: hah-bin.000004
          Read_Master_Log_Pos: 195
               Relay_Log_File: 82-relay-bin.000002
                Relay_Log_Pos: 365
        Relay_Master_Log_File: hah-bin.000004
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 195
              Relay_Log_Space: 570
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 80
                  Master_UUID: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

自動切換

(1)自動切換命令如下

[root@centos771 ~]# nohup masterha_manager --conf=/etc/masterha/app.cnf &> /dev/null & 
[1] 17335

[root@centos771 ~]# ps a
   PID TTY      STAT   TIME COMMAND
   602 tty1     Ss+    0:00 /sbin/agetty --noclear tty1 linux
 17044 pts/0    Ss     0:00 -bash
 17335 pts/0    S      0:00 perl /usr/bin/masterha_manager --conf=/etc/masterha/app.cnf
 17623 pts/0    R+     0:00 ps a

(2)然後手動關掉master

[root@hah ~]# systemctl stop mysqld

(3)然後會發現master轉到了server1上
server1的slave狀態為空

mysql> show slave status\G
Empty set (0.00 sec)

server2顯示master轉為server1

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.0.81
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: 81-bin.000002
          Read_Master_Log_Pos: 1407
               Relay_Log_File: 82-relay-bin.000002
                Relay_Log_Pos: 407
        Relay_Master_Log_File: 81-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1407
              Relay_Log_Space: 612
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 81
                  Master_UUID: 711d79aa-0eb8-11eb-b3dc-000c299dcf4d
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

(4)master需要開啟mysqld,然後新增master

[root@hah ~]# systemctl start mysqld


mysql> change master to master_host='10.0.0.81',master_port=3306,master_auto_position=1,master_user='centos',master_password='123456';
Query OK, 0 rows affected, 2 warnings (0.10 sec)

(5)master檢視slave的狀態,顯示master是server1,切換成功

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: 
                  Master_Host: 10.0.0.81
                  Master_User: centos
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: 
          Read_Master_Log_Pos: 4
               Relay_Log_File: hah-relay-bin.000001
                Relay_Log_Pos: 4
        Relay_Master_Log_File: 
             Slave_IO_Running: No
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 0
              Relay_Log_Space: 155
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 0
                  Master_UUID: 
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: 
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 2f33ae67-0eb8-11eb-b1ec-000c29f9b5e6:1-5
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
       Master_public_key_path: 
        Get_master_public_key: 0
1 row in set (0.00 sec)

VIP的漂移

1.編輯組態檔

[root@centos771 ~]# vim /etc/masterha/app.cnf
# [server default]新增如下一行
master_ip_failover_script=/usr/local/bin/master_ip_failover

2.下載MHA工具包,解壓安裝包

[root@centos771 ~]# cd /data/
[root@centos771 data]# wget https://github.com/yoshinorim/mha4mysql-manager/releases/download/v0.58/mha4mysql-manager-0.58.tar.gz
[root@centos771 data]# tar zxf mha4mysql-manager-0.58.tar.gz

# 查詢指令碼
[root@centos771 data]# find / -name "master_ip*"
/data/mha4mysql-manager-0.58/tests/t/master_ip_failover
/data/mha4mysql-manager-0.58/tests/t/master_ip_failover_blank
/data/mha4mysql-manager-0.58/samples/scripts/master_ip_failover
/data/mha4mysql-manager-0.58/samples/scripts/master_ip_online_change

3.在 /usr/local/bin新增1個檔案,並給1個檔案新增執行許可權

[root@centos771 data]# cd mha4mysql-manager-0.58/samples/scripts/
[root@centos771 scripts]# cp master_ip_
master_ip_failover       master_ip_online_change  
[root@centos771 scripts]# cp master_ip_failover /usr/local/bin/
[root@centos771 scripts]# cd /usr/local/bin
[root@centos771 bin]# ls
master_ip_failover

[root@centos771 bin]# vim master_ip_failover
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '10.0.0.100/24';#設定Virtual IP
my $gateway = '10.0.0.254';#閘道器Gateway IP
my $interface = 'ens33';    #指定VIP所在網路卡
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig $interface:$key $vip;/sbin/arping -I
$interface -c 3 -s $vip $gateway >/dev/null 2>&1";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {

my $exit_code = 1;
eval {
print "Disabling the VIP on old master: $orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {

my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}

sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}

sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --
orig_master_host=host --orig_master_ip=ip --orig_master_port=port --
new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}
[root@mha-manager ~]#chmod +x /usr/local/bin/master_ip_failover

4.在現在的master上(server1),新增VIP

[root@81 data]# ip addr add 10.0.0.100/24 dev ens33
[root@81 data]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:85:98:b8 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.81/8 brd 10.255.255.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 10.0.0.100/24 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe85:98b8/64 scope link 
       valid_lft forever preferred_lft forever

測試:自動轉換

# 檢查MHA的ssh
[root@771 data]# masterha_check_ssh --conf=/etc/masterha/app.cnf

# 檢查MHA環境
[root@771 data]# masterha_check_repl --conf=/etc/masterha/app.cnf

# 啟動MHA
[root@771 data]# masterha_manager --conf=/etc/masterha/app.cnf &> /dev/null &
[2] 2226
[root@771 data]# masterha_check_status --conf=/etc/masterha/app.cnf
app (pid:2160) is running(0:PING_OK), master:10.0.0.81
[2]+  Exit 1                  masterha_manager --conf=/etc/masterha/app.cnf &>/dev/null
# master停止服務
[root@80 data]# systemctl stop mysqld
檢視VIP漂移到server1上
# 檢視slave1的IP
[root@81 data]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:02:89:3c brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.81/8 brd 10.255.255.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 10.0.0.100/24 brd 10.0.0.255 scope global ens33:1
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe02:893c/64 scope link 
       valid_lft forever preferred_lft forever

4、實戰案例:Percona XtraDB Cluster(PXC 5.7)

注意:pxc目前支援最高的OS版本是centos7

1 環境準備

四臺虛擬機器器

  • pxc1:10.0.0.71
  • pxc2:10.0.0.72
  • pxc3:10.0.0.73
  • pxc4:10.0.0.74

**關閉防火牆和SELinux,保證時間同步 **

注意:如果已經安裝MySQL,必須解除安裝

2 安裝 Percona XtraDB Cluster 5.7

# 此處使用清華大學yum源,官方源太慢了
[root@71 ~]# cat > /etc/yum.repos.d/pxc.repo << EOF
[percona]
name=percona_repo
baseurl =https://mirrors.tuna.tsinghua.edu.cn/percona/release/\$releasever/RPMS/\$basearch
enabled = 1
gpgcheck = 0
EOF


[root@71 data]# scp /etc/yum.repos.d/pxc.repo 10.0.0.72:/etc/yum.repos.d/
[root@71 data]# scp /etc/yum.repos.d/pxc.repo 10.0.0.73:/etc/yum.repos.d/
[root@71 data]# scp /etc/yum.repos.d/pxc.repo 10.0.0.74:/etc/yum.repos.d/

#在三個節點都安裝好PXC 5.7 
[root@71 data]# yum -y install Percona-XtraDB-Cluster-57
[root@72 data]# yum -y install Percona-XtraDB-Cluster-57
[root@73 data]# yum -y install Percona-XtraDB-Cluster-57
[root@74 data]# yum -y install Percona-XtraDB-Cluster-57

3 在各個節點上分別設定mysql及叢集組態檔

/etc/my.cnf為主組態檔,當前版本中,其餘的組態檔都放在/etc/percona-xtradb-cluster.conf.d目 錄裡,包括mysqld.cnf,mysqld_safe.cnf,wsrep.cnf 三個檔案

pxc1

# 主組態檔不需要修改
[root@71 data]# cat /etc/my.cnf | grep -v "^#"
!includedir /etc/my.cnf.d/
!includedir /etc/percona-xtradb-cluster.conf.d/
[root@71 data]# ls /etc/my.cnf.
my.cnf.d/   my.cnf.old  
[root@71 data]# ls /etc/my.cnf.d/


[root@71 data]# ls /etc/percona-xtradb-cluster.conf.d/
mysqld.cnf  mysqld_safe.cnf  wsrep.cnf

# 下面組態檔不需要修改
[root@71 data]# cat /etc/percona-xtradb-cluster.conf.d/mysqld.cnf | grep -v "^#"
[client]
socket=/var/lib/mysql/mysql.sock

[mysqld]
server-id=71	 # 建議各個節點不同,使用IP
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-bin		 # 建議啟用,非必須項
log_slave_updates
expire_logs_days=7

symbolic-links=0

# 下面組態檔不需要修改
[root@71 data]# cat /etc/percona-xtradb-cluster.conf.d/mysqld_safe.cnf | grep -v "^#"

[mysqld_safe]
pid-file = /var/run/mysqld/mysqld.pid
socket   = /var/lib/mysql/mysql.sock
nice     = 0

# PXC的組態檔必須修改
[root@71 data]# vim /etc/percona-xtradb-cluster.conf.d/wsrep.cnf 
[root@71 data]# cat /etc/percona-xtradb-cluster.conf.d/wsrep.cnf | grep -v "^#"
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so

wsrep_cluster_address=gcomm://10.0.0.71,10.0.0.72,10.0.0.73	 #三個節點的IP

binlog_format=ROW

default_storage_engine=InnoDB

wsrep_slave_threads= 8

wsrep_log_conflicts

innodb_autoinc_lock_mode=2

wsrep_node_address=10.0.0.71	 #取消註釋,各個節點,指定自已的IP
wsrep_cluster_name=pxc-cluster

wsrep_node_name=pxc-cluster-node-1	#各個節點,指定自已節點名稱

pxc_strict_mode=ENFORCING

wsrep_sst_method=xtrabackup-v2

wsrep_sst_auth="sstuser:s3cretPass"	 #取消本行註釋

pxc2

[root@72 ~]# cat  /etc/percona-xtradb-cluster.conf.d/mysqld.cnf | grep -v "^#"
[client]
socket=/var/lib/mysql/mysql.sock

[mysqld]
server-id=72
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-bin
log_slave_updates
expire_logs_days=7

symbolic-links=0


[root@72 ~]# cat  /etc/percona-xtradb-cluster.conf.d/wsrep.cnf | grep -v "^#"
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so

wsrep_cluster_address=gcomm://10.0.0.71,10.0.0.72,10.0.0.73

binlog_format=ROW

default_storage_engine=InnoDB

wsrep_slave_threads= 8

wsrep_log_conflicts

innodb_autoinc_lock_mode=2

wsrep_node_address=10.0.0.72
wsrep_cluster_name=pxc-cluster

wsrep_node_name=pxc-cluster-node-2

pxc_strict_mode=ENFORCING

wsrep_sst_method=xtrabackup-v2

wsrep_sst_auth="sstuser:s3cretPass"

pxc3

[root@73 ~]# cat  /etc/percona-xtradb-cluster.conf.d/mysqld.cnf | grep -v "^#"
[client]
socket=/var/lib/mysql/mysql.sock

[mysqld]
server-id=73
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-bin
log_slave_updates
expire_logs_days=7

symbolic-links=0


[root@73 ~]# cat  /etc/percona-xtradb-cluster.conf.d/wsrep.cnf | grep -v "^#"
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so

wsrep_cluster_address=gcomm://10.0.0.71,10.0.0.72,10.0.0.73

binlog_format=ROW

default_storage_engine=InnoDB

wsrep_slave_threads= 8

wsrep_log_conflicts

innodb_autoinc_lock_mode=2

wsrep_node_address=10.0.0.73
wsrep_cluster_name=pxc-cluster

wsrep_node_name=pxc-cluster-node-3

pxc_strict_mode=ENFORCING

wsrep_sst_method=xtrabackup-v2

wsrep_sst_auth="sstuser:s3cretPass"		# 注意此處的使用者名稱,密碼

4 啟動PXC叢集中第一個節點

[root@71 data]# ss -ntul
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port              
tcp    LISTEN     0      128                 *:22                              *:*                  
tcp    LISTEN     0      128              [::]:22                           [::]:*                  

#啟動第一個節點
[root@71 data]# systemctl start mysql@bootstrap.service
[root@71 data]# ss -ntul
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port              
tcp    LISTEN     0      128                 *:22                              *:*                  
tcp    LISTEN     0      128                 *:4567                            *:*                  
tcp    LISTEN     0      80               [::]:3306                         [::]:*                  
tcp    LISTEN     0      128              [::]:22                           [::]:*           


#檢視root密碼
[root@71 data]# awk -F'root@localhost: ' '/temporary password/{print $2}' /var/log/mysqld.log 
0-4l.wwY*u_A

# 修改密碼
[root@71 data]# mysql -uroot -p0-4l.wwY*u_A
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.31-34-57-log

Copyright (c) 2009-2020 Percona LLC and/or its affiliates
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> alter user 'root'@'localhost' identified by '123456';
Query OK, 0 rows affected (0.01 sec)

mysql> select user,host,authentication_string from mysql.user;
+---------------+-----------+-------------------------------------------+
| user          | host      | authentication_string                     |
+---------------+-----------+-------------------------------------------+
| root          | localhost | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| mysql.session | localhost | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
| mysql.sys     | localhost | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
+---------------+-----------+-------------------------------------------+
3 rows in set (0.00 sec)

#建立相關使用者並授權:此處的使用者和密碼跟組態檔一致
mysql> GRANT RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT ON *.* TO
    -> 'sstuser'@'localhost'  IDENTIFIED BY 's3cretPass';
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> select user,host,authentication_string from mysql.user;
+---------------+-----------+-------------------------------------------+
| user          | host      | authentication_string                     |
+---------------+-----------+-------------------------------------------+
| root          | localhost | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| mysql.session | localhost | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
| mysql.sys     | localhost | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
| sstuser       | localhost | *1D3B4419453D37D70EC48955E49473559FF4778E |
+---------------+-----------+-------------------------------------------+
4 rows in set (0.00 sec)

# 檢視相關變數

mysql> show variables like 'wsrep%'\G
*************************** 1. row ***************************
Variable_name: wsrep_OSU_method
        Value: TOI
*************************** 2. row ***************************
Variable_name: wsrep_RSU_commit_timeout
        Value: 5000
*************************** 3. row ***************************
Variable_name: wsrep_auto_increment_control
        Value: ON
*************************** 4. row ***************************
Variable_name: wsrep_causal_reads
        Value: OFF
*************************** 5. row ***************************
Variable_name: wsrep_certification_rules
        Value: strict
*************************** 6. row ***************************
Variable_name: wsrep_certify_nonPK
        Value: ON
*************************** 7. row ***************************
Variable_name: wsrep_cluster_address
        Value: gcomm://10.0.0.71,10.0.0.72,10.0.0.73
*************************** 8. row ***************************
Variable_name: wsrep_cluster_name
        Value: pxc-cluster
*************************** 9. row ***************************
Variable_name: wsrep_convert_LOCK_to_trx
        Value: OFF
*************************** 10. row ***************************
Variable_name: wsrep_data_home_dir
        Value: /var/lib/mysql/
*************************** 11. row ***************************
Variable_name: wsrep_dbug_option
        Value: 
*************************** 12. row ***************************
Variable_name: wsrep_debug
        Value: OFF
*************************** 13. row ***************************
Variable_name: wsrep_desync
        Value: OFF
*************************** 14. row ***************************
Variable_name: wsrep_dirty_reads
        Value: OFF
*************************** 15. row ***************************
Variable_name: wsrep_drupal_282555_workaround
        Value: OFF
*************************** 16. row ***************************
Variable_name: wsrep_forced_binlog_format
        Value: NONE
*************************** 17. row ***************************
Variable_name: wsrep_load_data_splitting
        Value: ON
*************************** 18. row ***************************
Variable_name: wsrep_log_conflicts
        Value: ON
*************************** 19. row ***************************
Variable_name: wsrep_max_ws_rows
        Value: 0
*************************** 20. row ***************************
Variable_name: wsrep_max_ws_size
        Value: 2147483647
*************************** 21. row ***************************
Variable_name: wsrep_node_address
        Value: 10.0.0.71
*************************** 22. row ***************************
Variable_name: wsrep_node_incoming_address
        Value: AUTO
*************************** 23. row ***************************
Variable_name: wsrep_node_name
        Value: pxc-cluster-node-1
*************************** 24. row ***************************
Variable_name: wsrep_notify_cmd
        Value: 
*************************** 25. row ***************************
Variable_name: wsrep_on
        Value: ON
*************************** 26. row ***************************
Variable_name: wsrep_preordered
        Value: OFF
*************************** 27. row ***************************
Variable_name: wsrep_provider
        Value: /usr/lib64/galera3/libgalera_smm.so
*************************** 28. row ***************************
Variable_name: wsrep_provider_options
        Value: base_dir = /var/lib/mysql/; base_host = 10.0.0.71; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.causal_keepalive_period = PT1S; evs.debug_log_mask = 0x1; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.info_log_mask = 0; evs.install_timeout = PT7.5S; evs.join_retrans_period = PT1S; evs.keepalive_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 10; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.use_aggregate = true; evs.user_send_window = 4; evs.version = 0; evs.view_forget_timeout = P1D; gcache.dir = /var/lib/mysql/; gcache.freeze_purge_at_seqno = -1; gcache.keep_pages_count = 0; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 100; gcs.fc_master_slave = no; gc
*************************** 29. row ***************************
Variable_name: wsrep_recover
        Value: OFF
*************************** 30. row ***************************
Variable_name: wsrep_reject_queries
        Value: NONE
*************************** 31. row ***************************
Variable_name: wsrep_replicate_myisam
        Value: OFF
*************************** 32. row ***************************
Variable_name: wsrep_restart_slave
        Value: OFF
*************************** 33. row ***************************
Variable_name: wsrep_retry_autocommit
        Value: 1
*************************** 34. row ***************************
Variable_name: wsrep_slave_FK_checks
        Value: ON
*************************** 35. row ***************************
Variable_name: wsrep_slave_UK_checks
        Value: OFF
*************************** 36. row ***************************
Variable_name: wsrep_slave_threads
        Value: 8
*************************** 37. row ***************************
Variable_name: wsrep_sst_auth
        Value: ********
*************************** 38. row ***************************
Variable_name: wsrep_sst_donor
        Value: 
*************************** 39. row ***************************
Variable_name: wsrep_sst_donor_rejects_queries
        Value: OFF
*************************** 40. row ***************************
Variable_name: wsrep_sst_method
        Value: xtrabackup-v2
*************************** 41. row ***************************
Variable_name: wsrep_sst_receive_address
        Value: AUTO
*************************** 42. row ***************************
Variable_name: wsrep_start_position
        Value: 00000000-0000-0000-0000-000000000000:-1
*************************** 43. row ***************************
Variable_name: wsrep_sync_wait
        Value: 0
43 rows in set (0.01 sec)
#檢視相關狀態變數
mysql> show status like 'wsrep%'\G
*************************** 1. row ***************************
Variable_name: wsrep_local_state_uuid
        Value: 19eed63b-0fb0-11eb-97c4-4f01a0c74176
*************************** 2. row ***************************
Variable_name: wsrep_protocol_version
        Value: 9
*************************** 3. row ***************************
Variable_name: wsrep_last_applied
        Value: 2
*************************** 4. row ***************************
Variable_name: wsrep_last_committed
        Value: 2
*************************** 5. row ***************************
Variable_name: wsrep_replicated
        Value: 2
*************************** 6. row ***************************
Variable_name: wsrep_replicated_bytes
        Value: 544
*************************** 7. row ***************************
Variable_name: wsrep_repl_keys
        Value: 2
*************************** 8. row ***************************
Variable_name: wsrep_repl_keys_bytes
        Value: 64
*************************** 9. row ***************************
Variable_name: wsrep_repl_data_bytes
        Value: 348
*************************** 10. row ***************************
Variable_name: wsrep_repl_other_bytes
        Value: 0
*************************** 11. row ***************************
Variable_name: wsrep_received
        Value: 2
*************************** 12. row ***************************
Variable_name: wsrep_received_bytes
        Value: 151
*************************** 13. row ***************************
Variable_name: wsrep_local_commits
        Value: 0
*************************** 14. row ***************************
Variable_name: wsrep_local_cert_failures
        Value: 0
*************************** 15. row ***************************
Variable_name: wsrep_local_replays
        Value: 0
*************************** 16. row ***************************
Variable_name: wsrep_local_send_queue
        Value: 0
*************************** 17. row ***************************
Variable_name: wsrep_local_send_queue_max
        Value: 1
*************************** 18. row ***************************
Variable_name: wsrep_local_send_queue_min
        Value: 0
*************************** 19. row ***************************
Variable_name: wsrep_local_send_queue_avg
        Value: 0.000000
*************************** 20. row ***************************
Variable_name: wsrep_local_recv_queue
        Value: 0
*************************** 21. row ***************************
Variable_name: wsrep_local_recv_queue_max
        Value: 2
*************************** 22. row ***************************
Variable_name: wsrep_local_recv_queue_min
        Value: 0
*************************** 23. row ***************************
Variable_name: wsrep_local_recv_queue_avg
        Value: 0.500000
*************************** 24. row ***************************
Variable_name: wsrep_local_cached_downto
        Value: 1
*************************** 25. row ***************************
Variable_name: wsrep_flow_control_paused_ns
        Value: 0
*************************** 26. row ***************************
Variable_name: wsrep_flow_control_paused
        Value: 0.000000
*************************** 27. row ***************************
Variable_name: wsrep_flow_control_sent
        Value: 0
*************************** 28. row ***************************
Variable_name: wsrep_flow_control_recv
        Value: 0
*************************** 29. row ***************************
Variable_name: wsrep_flow_control_interval
        Value: [ 100, 100 ]
*************************** 30. row ***************************
Variable_name: wsrep_flow_control_interval_low
        Value: 100
*************************** 31. row ***************************
Variable_name: wsrep_flow_control_interval_high
        Value: 100
*************************** 32. row ***************************
Variable_name: wsrep_flow_control_status
        Value: OFF
*************************** 33. row ***************************
Variable_name: wsrep_cert_deps_distance
        Value: 1.000000
*************************** 34. row ***************************
Variable_name: wsrep_apply_oooe
        Value: 0.000000
*************************** 35. row ***************************
Variable_name: wsrep_apply_oool
        Value: 0.000000
*************************** 36. row ***************************
Variable_name: wsrep_apply_window
        Value: 1.000000
*************************** 37. row ***************************
Variable_name: wsrep_commit_oooe
        Value: 0.000000
*************************** 38. row ***************************
Variable_name: wsrep_commit_oool
        Value: 0.000000
*************************** 39. row ***************************
Variable_name: wsrep_commit_window
        Value: 1.000000
*************************** 40. row ***************************
Variable_name: wsrep_local_state
        Value: 4
*************************** 41. row ***************************
Variable_name: wsrep_local_state_comment
        Value: Synced
*************************** 42. row ***************************
Variable_name: wsrep_cert_index_size
        Value: 1
*************************** 43. row ***************************
Variable_name: wsrep_cert_bucket_count
        Value: 22
*************************** 44. row ***************************
Variable_name: wsrep_gcache_pool_size
        Value: 1944
*************************** 45. row ***************************
Variable_name: wsrep_causal_reads
        Value: 0
*************************** 46. row ***************************
Variable_name: wsrep_cert_interval
        Value: 0.000000
*************************** 47. row ***************************
Variable_name: wsrep_open_transactions
        Value: 0
*************************** 48. row ***************************
Variable_name: wsrep_open_connections
        Value: 0
*************************** 49. row ***************************
Variable_name: wsrep_ist_receive_status
        Value: 
*************************** 50. row ***************************
Variable_name: wsrep_ist_receive_seqno_start
        Value: 0
*************************** 51. row ***************************
Variable_name: wsrep_ist_receive_seqno_current
        Value: 0
*************************** 52. row ***************************
Variable_name: wsrep_ist_receive_seqno_end
        Value: 0
*************************** 53. row ***************************
Variable_name: wsrep_incoming_addresses
        Value: 10.0.0.71:3306
*************************** 54. row ***************************
Variable_name: wsrep_cluster_weight
        Value: 1
*************************** 55. row ***************************
Variable_name: wsrep_desync_count
        Value: 0
*************************** 56. row ***************************
Variable_name: wsrep_evs_delayed
        Value: 
*************************** 57. row ***************************
Variable_name: wsrep_evs_evict_list
        Value: 
*************************** 58. row ***************************
Variable_name: wsrep_evs_repl_latency
        Value: 0/0/0/0/0
*************************** 59. row ***************************
Variable_name: wsrep_evs_state
        Value: OPERATIONAL
*************************** 60. row ***************************
Variable_name: wsrep_gcomm_uuid
        Value: 19ec9903-0fb0-11eb-956e-3e998fb96c10
*************************** 61. row ***************************
Variable_name: wsrep_cluster_conf_id
        Value: 1
*************************** 62. row ***************************
Variable_name: wsrep_cluster_size
        Value: 1
*************************** 63. row ***************************
Variable_name: wsrep_cluster_state_uuid
        Value: 19eed63b-0fb0-11eb-97c4-4f01a0c74176
*************************** 64. row ***************************
Variable_name: wsrep_cluster_status
        Value: Primary
*************************** 65. row ***************************
Variable_name: wsrep_connected
        Value: ON
*************************** 66. row ***************************
Variable_name: wsrep_local_bf_aborts
        Value: 0
*************************** 67. row ***************************
Variable_name: wsrep_local_index
        Value: 0
*************************** 68. row ***************************
Variable_name: wsrep_provider_name
        Value: Galera
*************************** 69. row ***************************
Variable_name: wsrep_provider_vendor
        Value: Codership Oy <info@codership.com>
*************************** 70. row ***************************
Variable_name: wsrep_provider_version
        Value: 3.45(ra60e019)
*************************** 71. row ***************************
Variable_name: wsrep_ready
        Value: ON
71 rows in set (0.00 sec)
#重點關注下面內容
mysql> show status like 'wsrep%';
+----------------------------------+--------------------------------------+
| Variable_name                    | Value                                |
+----------------------------------+--------------------------------------+
| wsrep_local_state_uuid           | 19eed63b-0fb0-11eb-97c4-4f01a0c74176 |
.............
.............
| wsrep_local_state                | 4                                    |
| wsrep_local_state_comment        | Synced                               |
.............
| wsrep_cluster_size               | 1                                    |
| wsrep_cluster_state_uuid         | 19eed63b-0fb0-11eb-97c4-4f01a0c74176 |
| wsrep_cluster_status             | Primary                              |
| wsrep_connected                  | ON                                   |
..............
| wsrep_ready                      | ON                                   |
+----------------------------------+--------------------------------------+
71 rows in set (0.00 sec)

說明:

  • wsrep_cluster_size表示,該Galera叢集中只有一個節點
  • wsrep_local_state_comment 狀態為Synced(4),表示資料已同步完成(因為是第一個引導節點,無數 據需要同步)。 如果狀態是Joiner, 意味著 SST 沒有完成. 只有所有節點狀態是Synced,才可以加新節點
  • wsrep_cluster_status為Primary,且已經完全連線並準備好

5 啟動PXC叢集中其它所有節點

pxc2

[root@72 ~]# ss -ntul
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port              
tcp    LISTEN     0      128                 *:22                              *:*                  
tcp    LISTEN     0      128              [::]:22                           [::]:*                  
[root@72 ~]# systemctl start mysql
[root@72 ~]# ss -ntulp
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port              
tcp    LISTEN     0      128                 *:22                              *:*                   users:(("sshd",pid=804,fd=3))
tcp    LISTEN     0      128                 *:4567                            *:*                   users:(("mysqld",pid=2029,fd=11))
tcp    LISTEN     0      80               [::]:3306                         [::]:*                   users:(("mysqld",pid=2029,fd=34))
tcp    LISTEN     0      128              [::]:22                           [::]:*                   users:(("sshd",pid=804,fd=4))

pxc3

[root@73 ~]# ss -ntul
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port              
tcp    LISTEN     0      128                 *:22                              *:*                  
tcp    LISTEN     0      128              [::]:22                           [::]:*                  
[root@73 ~]# systemctl start mysql

[root@73 ~]# ss -ntulp
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port              
tcp    LISTEN     0      128                 *:22                              *:*                   users:(("sshd",pid=808,fd=3))
tcp    LISTEN     0      128                 *:4567                            *:*                   users:(("mysqld",pid=1954,fd=11))
tcp    LISTEN     0      128              [::]:22                           [::]:*                   users:(("sshd",pid=808,fd=4))
tcp    LISTEN     0      80               [::]:3306                         [::]:*                   users:(("mysqld",pid=1954,fd=34))

6 檢視叢集狀態,驗證叢集是否成功

pxc1

mysql> show variables like 'wsrep_node_name';
+-----------------+--------------------+
| Variable_name   | Value              |
+-----------------+--------------------+
| wsrep_node_name | pxc-cluster-node-1 |
+-----------------+--------------------+
1 row in set (0.00 sec)

mysql> show variables like 'wsrep_node_address';
+--------------------+-----------+
| Variable_name      | Value     |
+--------------------+-----------+
| wsrep_node_address | 10.0.0.71 |
+--------------------+-----------+
1 row in set (0.00 sec)

mysql> show variables like 'wsrep_on';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| wsrep_on      | ON    |
+---------------+-------+
1 row in set (0.00 sec)


mysql> SHOW STATUS LIKE 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 3     |
+--------------------+-------+
1 row in set (0.00 sec)

7 測試

#在任意節點建立資料庫
# pxc1
mysql> create database db1;
Query OK, 1 row affected (0.00 sec)

# pxc2
# 注意:登入密碼為pxc1修改的密碼
[root@72 ~]# mysql -uroot -p123456;history -c
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 5.7.31-34-57-log Percona XtraDB Cluster (GPL), Release rel34, Revision 7359e4f, WSREP version 31.45, wsrep_31.45

Copyright (c) 2009-2020 Percona LLC and/or its affiliates
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)


# pxc3
[root@72 ~]# mysql -uroot -p123456;history -c
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 14
Server version: 5.7.31-34-57-log Percona XtraDB Cluster (GPL), Release rel34, Revision 7359e4f, WSREP version 31.45, wsrep_31.45

Copyright (c) 2009-2020 Percona LLC and/or its affiliates
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)
#利用Xshell軟體,同時在三個節點資料庫建立資料庫,在其中一個節點成功

# pxc2
mysql> create database db2;
ERROR 1007 (HY000): Can't create database 'db2'; database exists
mysql> 

# pxc1
mysql> create database db2;
Query OK, 1 row affected (0.01 sec)

# pxc3
mysql> create database db2;
ERROR 1007 (HY000): Can't create database 'db2'; database exists

8 在PXC叢集中加入節點

一個節點加入到Galera叢集有兩種情況:新節點加入叢集、暫時離組的成員再次加入叢集

1)新節點加入Galera叢集

​ 新節點加入叢集時,需要從當前叢集中選擇一個Donor節點來同步資料,也就是所謂的 state_snapshot_tranfer(SST)過程。SST同步資料的方式由選項wsrep_sst_method決定,一般選擇的 是xtrabackup。 必須注意,新節點加入Galera時,會刪除新節點上所有已有資料,再通過xtrabackup(假設使用的是該 方式)從Donor處完整備份所有資料進行恢復。所以,如果資料量很大,新節點加入過程會很慢。而且, 在一個新節點成為Synced狀態之前,不要同時加入其它新節點,否則很容易將叢集壓垮。 如果是這種情況,可以考慮使用wsrep_sst_method=rsync來做增量同步,既然是增量同步,最好保證 新節點上已經有一部分資料基礎,否則和全量同步沒什麼區別,且這樣會對Donor節點加上全域性read only鎖。

2)舊節點加入Galera叢集

​ 如果舊節點加入Galera叢集,說明這個節點在之前已經在Galera叢集中呆過,有一部分資料基礎,缺少 的只是它離開叢集時的資料。這時加入叢集時,會採用IST(incremental snapshot transfer)傳輸機制, 即使用增量傳輸。 但注意,這部分增量傳輸的資料來源是Donor上快取在GCache檔案中的,這個檔案有大小限制,如果缺 失的資料範圍超過已快取的內容,則自動轉為SST傳輸。如果舊節點上的資料和Donor上的資料不匹配 (例如這個節點離組後人為修改了一點資料),則自動轉為SST傳輸。

#在PXC叢集中再加一臺新的主機PXC4:10.0.0.37

[root@74 ~]# cat /etc/percona-xtradb-cluster.conf.d/mysqld.cnf | grep -v "^#"
[client]
socket=/var/lib/mysql/mysql.sock

[mysqld]
server-id=74
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-bin
log_slave_updates
expire_logs_days=7

symbolic-links=0
[root@74 ~]# cat /etc/percona-xtradb-cluster.conf.d/wsrep.cnf | grep -v "^#"
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so

wsrep_cluster_address=gcomm://10.0.0.71,10.0.0.72,10.0.0.73,10.0.0.74

binlog_format=ROW

default_storage_engine=InnoDB

wsrep_slave_threads= 8

wsrep_log_conflicts

innodb_autoinc_lock_mode=2

wsrep_node_address=10.0.0.74
wsrep_cluster_name=pxc-cluster

wsrep_node_name=pxc-cluster-node-4

pxc_strict_mode=ENFORCING

wsrep_sst_method=xtrabackup-v2

wsrep_sst_auth="sstuser:s3cretPass"
# 啟動服務
[root@74 ~]# systemctl restart mysql
[root@74 ~]# ss -ntulp
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port              
tcp    LISTEN     0      128                 *:22                              *:*                   users:(("sshd",pid=807,fd=3))
tcp    LISTEN     0      128                 *:4567                            *:*                   users:(("mysqld",pid=15095,fd=11))
tcp    LISTEN     0      128              [::]:22                           [::]:*                   users:(("sshd",pid=807,fd=4))
tcp    LISTEN     0      80               [::]:3306                         [::]:*                   users:(("mysqld",pid=15095,fd=35))


# 登入資料庫
[root@74 ~]# mysql -uroot -p123456
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.31-34-57-log Percona XtraDB Cluster (GPL), Release rel34, Revision 7359e4f, WSREP version 31.45, wsrep_31.45

Copyright (c) 2009-2020 Percona LLC and/or its affiliates
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show status like 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 4     |
+--------------------+-------+
1 row in set (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| db2                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
6 rows in set (0.00 sec)
# 將其它節點的組態檔加以修改
# pxc3
[root@73 ~]# sed -ri '/^wsrep_cluster_address/s#(.*)#\1,10.0.0.74#' /etc/percona-xtradb-cluster.conf.d/wsrep.cnf 
[root@73 ~]# cat  /etc/percona-xtradb-cluster.conf.d/wsrep.cnf | grep -v "^#"
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so

wsrep_cluster_address=gcomm://10.0.0.71,10.0.0.72,10.0.0.73,10.0.0.74

binlog_format=ROW

default_storage_engine=InnoDB

wsrep_slave_threads= 8

wsrep_log_conflicts

innodb_autoinc_lock_mode=2

wsrep_node_address=10.0.0.73
wsrep_cluster_name=pxc-cluster

wsrep_node_name=pxc-cluster-node-3

pxc_strict_mode=ENFORCING

wsrep_sst_method=xtrabackup-v2

wsrep_sst_auth="sstuser:s3cretPass"




# pxc2
[root@72 ~]# sed -ri '/^wsrep_cluster_address/s#(.*)#\1,10.0.0.74#' /etc/percona-xtradb-cluster.conf.d/wsrep.cnf

# pxc1
[root@71 data]# sed -ri '/^wsrep_cluster_address/s#(.*)#\1,10.0.0.74#' /etc/percona-xtradb-cluster.conf.d/wsrep.cnf

9 在PXC叢集中修復故障節點

#在任意節點停止服務

# pxc3
[root@73 ~]# systemctl stop mysql

# pxc2
mysql> show status like 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 3     |
+--------------------+-------+
1 row in set (0.01 sec)

# pxc1
mysql> show status like 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 3     |
+--------------------+-------+
1 row in set (0.00 sec)


# 建立資料庫
# pxc4
mysql> create database db3;
Query OK, 1 row affected (0.00 sec)

# pxc2
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| db2                |
| db3                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
7 rows in set (0.00 sec)


# pxc3開啟服務
[root@73 ~]# systemctl start mysql
[root@73 ~]# mysql -uroot -p123456
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.31-34-57-log Percona XtraDB Cluster (GPL), Release rel34, Revision 7359e4f, WSREP version 31.45, wsrep_31.45

Copyright (c) 2009-2020 Percona LLC and/or its affiliates
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| db2                |
| db3                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
7 rows in set (0.00 sec)

# 資料已同步

5、通過 ansible 部署二進位制 mysql 8

實驗環境準備

虛擬機器器IP
centos8110.0.0.81ansible伺服器主控端
centos8210.0.0.82遠端主機1
centos8310.0.0.83遠端主機2

前期準備

1、關閉防火牆:systemctl disable --now firewalld
2、關閉selinux安全機制: sed -ri 's/(^SELINUX=).*/\1disabled/' /etc/selinux/config
3、重新啟動生效:init 6

1、安裝ansible

使用EPEL源的rpm包安裝

# 使用阿里雲映象站的eple源:https://developer.aliyun.com/mirror/epel?spm=a2c6h.13651102.0.0.3e221b11pyj3r4
# 1、如果之前設定了epel源,則先備份;如果沒有就跳過這步
[root@81 ~]# mv /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel.repo.backup

[root@81 ~]# mv /etc/yum.repos.d/epel-testing.repo /etc/yum.repos.d/epel-testing.repo.backup

# 2. 下載新repo 到/etc/yum.repos.d/
1)安裝 epel 設定包
[root@81 ~]# yum install -y https://mirrors.aliyun.com/epel/epel-release-latest-8.noarch.rpm

2)將 repo 設定中的地址替換為阿里雲映象站地址
[root@81 ~]# sed -i 's|^#baseurl=https://download.fedoraproject.org/pub|baseurl=https://mirrors.aliyun.com|' /etc/yum.repos.d/epel*
[root@81 ~]# sed -i 's|^metalink|#metalink|' /etc/yum.repos.d/epel*


# 安裝ansible
[root@81 ~]# yum -y install ansible
[root@81 ~]# ansible --version
ansible 2.9.14
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

2、修改ansible組態檔

# /etc/ansible/ansible.cfg
[root@81 data]# vim /etc/ansible/ansible.cfg 
[defaults]
host_key_checking = False	# 解開註釋
# /etc/ansible/hosts
# 直接設定ssh連線,使用者名稱,密碼,執行yaml指令碼就不需要加 -k 
[root@81 data]# vim /etc/ansible/hosts
[webservser]
10.0.0.81  ansible_connection=local
10.0.0.82  ansible_connection=ssh   ansible_user=root  ansible_password=123456
10.0.0.83  ansible_connection=ssh   ansible_user=root  ansible_password=123456

3、準備mysql相關檔案

# my.cnf 組態檔
[root@81 data]# cat my.cnf 
[mysqld]
# 指定資料庫資料存放路徑
datadir=/data/mysql

# 指定套間字路徑
socket=/data/mysql/mysql.sock

# 設定紀錄檔檔案路徑
log-error=/data/mysql/mysqld.log

#關閉DNS反向解析
skip_name_resolve=on
port=3306
#pid-file=/tmp/mysql.pid

[client]
port=3306
# 指定套間字路徑
socket=/data/mysql/mysql.sock
# passwd.sh 修改mysql密碼
[root@81 data]# vim passwd.sh
pwd=`awk '/temporary password/{print $NF}' /data/mysql/mysqld.log`
mysqladmin -uroot -p"$pwd" password 12345678

4、編輯yaml指令碼

[root@81 data]# vim install_mysql.yml 
- hosts: all 
  remote_user: root
  gather_facts: no
  vars:
  - path: https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.21-linux-glibc2.17-x86_64-minimal.tar.xz
  
  tasks:
    - name: "關閉初次存取提示詢問"
      shell: sed -i "s/^.*StrictHostKeyChecking.*$/   StrictHostKeyChecking no/g" /etc/ssh/ssh_config
    - name: "刪除ssh檔案"
      file: path=/root/.ssh/ state=absent
    - name: "生成公鑰私鑰對"
      shell: ssh-keygen -t rsa -b 2048 -N '' -f /root/.ssh/id_rsa
    - name: "刪除臨時ssh目錄"
      file: path=/tmp/ssh/ state=absent
      run_once: true
    - name: "從各宿主機將公鑰拷貝到本機"
      fetch: src=/root/.ssh/id_rsa.pub dest=/tmp/ssh/
    - name: "將各個公鑰合併成一個檔案"
      shell: find /tmp/ssh/* -type f -exec sh -c 'cat {}>>/tmp/ssh/authorized_keys.log' \;
      run_once: true
    - name: "將合成的公鑰進行分發"
      copy: src=/tmp/ssh/authorized_keys.log dest=/root/.ssh/authorized_keys mode=0600
      tags:
        - install ssh  

    - name: "安裝依賴庫"
      yum: name=libaio,ncurses-compat-libs
    - name: "建立組"
      group: name=mysql gid=306
    - name: "建立使用者"
      user: name=mysql uid=306 group=mysql shell=/sbin/nologin system=yes create_home=no home=/data/mysql
    - name: "建立資料檔案"
      file: path=/data/mysql state=directory owner=mysql group=mysql recurse=yes
    - name: "下載wget"
      yum: name=wget state=latest
    - name: "下載mysql8.0二進位制安裝包"
      shell: wget -P /data/ {{ path }}
    - name: "解壓縮包到指定目錄:/use/local/"
      shell: find /data/ -name "*8.0*" | awk -F"/" "{print $3}" | xargs -I {} tar xf {} -C /usr/local/  
    - name: "建立MySQL軟連線"
      shell: find /usr/local/ -maxdepth 1 -type d -name "mysql*" -exec ln -s {} /usr/local/mysql \;
    - name: "修改連線許可權"
      file: dest=/usr/local/mysql owner=root group=root recurse=yes state=directory
    
    - name: "修改MySQL組態檔"
      copy: src=/data/my.cnf  dest=/etc/ backup=yes
    - name: "初始化資料庫"
      shell: chdir=/usr/local/mysql bin/mysqld --initialize --datadir=/data/mysql --user=mysql

    - name: "將mysql新增到服務"
      copy: src=/usr/local/mysql/support-files/mysql.server dest=/etc/init.d/mysqld
    - name: "設定開機自啟"
      shell: chkconfig --add mysqld;chkconfig mysqld on
    - name: "建立mysql執行軟連線"
      shell: ln -s /usr/local/mysql/bin/* /usr/bin/
	- name: secure script
      script: /data/passwd.sh
      tags: script
    
    - name: "啟動mysql服務"
      service: name=mysqld state=started

5、執行指令碼

[root@81 data]# ansible-playbook install_mysql.yml
[root@81 ~]# ansible-playbook /data/install_mysql.yml 

PLAY [all] *****************************************************************************************

TASK [關閉初次存取提示詢問] **********************************************************************************
[WARNING]: Consider using the replace, lineinfile or template module rather than running 'sed'.  If
you need to use command because replace, lineinfile or template is insufficient you can add 'warn:
false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this
message.
changed: [10.0.0.81]
changed: [10.0.0.82]
changed: [10.0.0.83]

TASK [刪除ssh檔案] *************************************************************************************
changed: [10.0.0.81]
ok: [10.0.0.82]
ok: [10.0.0.83]

TASK [生成公鑰私鑰對] *************************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.82]
changed: [10.0.0.83]

TASK [刪除臨時ssh目錄] ***********************************************************************************
ok: [10.0.0.81]

TASK [從各宿主機將公鑰拷貝到本機] *******************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [將各個公鑰合併成一個檔案] ********************************************************************************
changed: [10.0.0.81]

TASK [將合成的公鑰進行分發] **********************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.82]
changed: [10.0.0.83]

TASK [安裝依賴庫] ***************************************************************************************
changed: [10.0.0.83]
changed: [10.0.0.81]
changed: [10.0.0.82]

TASK [建立組] *****************************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.82]
changed: [10.0.0.83]

TASK [建立使用者] ****************************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.82]
changed: [10.0.0.83]

TASK [建立資料檔案] **************************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [下載wget] **************************************************************************************
ok: [10.0.0.81]
ok: [10.0.0.82]
ok: [10.0.0.83]

TASK [下載mysql8.0二進位制安裝包] ****************************************************************************
[WARNING]: Consider using the get_url or uri module rather than running 'wget'.  If you need to use
command because get_url or uri is insufficient you can add 'warn: false' to this command task or
set 'command_warnings=False' in ansible.cfg to get rid of this message.
changed: [10.0.0.83]
changed: [10.0.0.81]
changed: [10.0.0.82]

TASK [解壓縮包到指定目錄:/use/local/] ***********************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [建立MySQL軟連線] **********************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [修改連線許可權] **************************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [修改MySQL組態檔] *********************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [初始化資料庫] **************************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [將mysql新增到服務] *********************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [設定開機自啟] **************************************************************************************
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [建立mysql執行軟連線] ********************************************************************************
[WARNING]: Consider using the file module with state=link rather than running 'ln'.  If you need to
use command because file is insufficient you can add 'warn: false' to this command task or set
'command_warnings=False' in ansible.cfg to get rid of this message.
changed: [10.0.0.81]
changed: [10.0.0.83]
changed: [10.0.0.82]

TASK [啟動mysql服務] ***********************************************************************************
changed: [10.0.0.82]
changed: [10.0.0.83]
changed: [10.0.0.81]

PLAY RECAP *****************************************************************************************
10.0.0.81                  : ok=22   changed=20   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.82                  : ok=20   changed=18   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
10.0.0.83                  : ok=20   changed=18   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0