Ansible 安裝 Redis Cluster

之前筆記 Ansible 安裝 Redis Cluster 紀錄了以 ansible 內建 function 為主的 redis cluster 安裝 script,最近因為需要將 redis5 更新至 redis6,重新 review 了 script,做了些調整與優化,紀錄一下

基本環境說明

  1. Azure VM B2s (2 vcpu,4GiB memory)
  2. OpenLogic 7.7
  3. Redis 6.0.6
  4. ansible 2.9.11

安裝語法

  1. 專案結構

    • roles
      • redis-cluster
        • defaults
          • main.yml
        • handlers
          • main.yml
        • tasks
          • checkdir.yml
          • createcluster.yml
          • installredis.yml
          • installtools.yml
          • main.yml
          • prepareservice.yml
          • prepareredisconfig.yml
          • uninstall.yml
          • upgrade.yml
        • templates
          • config.j2
          • service.j2
        • vars
          • main.yml
        • inventories
          • dev.ini
        • README.md
    • ansible.cfg
    • main.yml
  2. 完整內容

    • roles

      • redis-cluster

        • defaults

          • main.yml
            ---
            purpose: update
          
        • handlers

          • main.yml
            ---
            - name: "Restart Redis Service"
              listen: restart-service
              systemd:
                name: "redis_{{item}}"
                daemon_reload: yes
                enabled: yes
                state: restarted
              with_items: "{{redis_ports}}"
          
        • tasks

          • checkdir.yml
            ---
            - name: Ensures dir exists
              file: 
                path: "{{ folder }}"
                state: directory
                mode: 0755
                owner: redis
                group: redis
                recurse: yes
          
          • createcluster.yml
            ---
            - name: Create Cluster debug
              debug:
                msg: redis-cli -a {{redispass}} --cluster-replicas 1 --cluster create {% for redis_node in groups['redis_all'] %} {% for redis_port in hostvars[redis_node].redis_ports %} {{ hostvars[redis_node].ip }}:{{ redis_port }} {% endfor %}{% endfor %}
          
            - name: Create Cluster
              shell: |
                echo "yes" |redis-cli -a {{redispass}} --cluster-replicas 1 --cluster create {% for redis_node in groups['redis_all'] %} {% for redis_port in hostvars[redis_node].redis_ports %} {{ hostvars[redis_node].ip }}:{{ redis_port }} {% endfor %}{% endfor %}
          
          • installredis.yml
            ---
            - name: Install Redis
              yum:
                name: "{{redisversion}}"
                state: latest
          
          • installtools.yml
            - name: Get rpm
              shell: wget http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
          
            - name: Install rpm
              ignore_errors: yes
              shell: rpm -Uvh remi-release-7.rpm
          
            - name: Enable remi repo
              shell: sed -i 's/enabled=0/enabled=1/g' /etc/yum.repos.d/remi.repo
          
            - name: Insall Redis
              include_tasks: installredis.yml
          
            - name: check dir
              include_tasks: checkdir.yml
              vars:
                folder: "/etc/redis"
          
            - name: Remove default redis service
              ignore_errors: yes
              shell: |
                systemctl disable redis 
          
            - name: Delete redis service
              ignore_errors: yes
              file: 
                path: "{{ item }}"
                state: absent
              with_items: 
                - /usr/lib/systemd/system/redis.service
                - /etc/systemd/system/redis.service.d
                - /etc/systemd/system/redis-sentinel.service.d
          
            - name: Disable SELinux
              shell: |
                setenforce 0
                sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
          
            - name: Allow Overcommit Memory
              sysctl:
                name: vm.overcommit_memory
                value: "1"
                state: present
                reload: yes
                ignoreerrors: yes
          
            - name: Stop and disable firewalld.
              service:
                name: firewalld
                state: stopped
                enabled: False
          
          • main.yml
            ---
            - name: Uninstall
              include_tasks: uninstall.yml
              when: purpose == 'uninstall' or purpose == 'reinstall'
          
            - name: Install rpm and redis
              include_tasks: installtools.yml
              when: purpose == 'install'
          
            - name: Upgrde redis
              include_tasks: upgrade.yml
              when: purpose == 'upgrade'
          
            - name: create linux user
              shell: |
                groupadd --system redis
                useradd -s /sbin/nologin --system -g redis redis
              ignore_errors: yes
              when: purpose == 'install'
          
            - name: Generate redis config file
              include_tasks: prepareredisconfig.yml
              when: purpose != 'uninstall'
          
            - name: Generate service file
              include_tasks: prepareservice.yml
              when: purpose == 'install' or purpose == 'reinstall'
          
            - name: Restart Service
              debug:
                msg: restart service
              notify: restart-service
              changed_when: true
              when: purpose == 'restart'
          
            - meta: flush_handlers
          
            - name: Run cluster creation script
              include_tasks: createcluster.yml
              run_once: true
              when: purpose == 'install' or purpose == 'reinstall'
          
          • prepareredisconfig.yml
            ---
            - name: check dir - /etc/redis/redis_{{ item }}
              include_tasks: checkdir.yml
              vars:
                folder: "/etc/redis/redis_{{ item }}"
              with_items: "{{ redis_ports }}"
          
            - name: Prepare Configs
              vars:
                redis_port: "{{item}}"
              template:
                src: config.j2
                dest: "/etc/redis/redis_{{ item }}.conf"
                owner: redis
              notify: restart-service
              with_items:  "{{redis_ports}}"
          
          • prepareservice.yml
            ---
            - name: Prepare services
              vars:
                redis_port: "{{item}}"
              template:
                src: service.j2
                dest: "/etc/systemd/system/redis_{{item}}.service"
                owner: redis
                group: redis
              with_items: "{{redis_ports}}"
          
          • uninstall.yml
            - name: Stop Service
                service:
                  name: "redis_{{item}}"
                  state: stopped
                with_items: "{{ redis_ports }}"
          
              - name: Disable Service
                shell: systemctl disable redis_{{item}}
                with_items: "{{ redis_ports }}"
          
              - name: Delete config
                file:
                  path: "/etc/redis/redis_{{item}}.conf"
                  state: absent
                with_items: "{{ redis_ports }}"
          
              - name: Delete service
                file:
                  path: "/etc/systemd/system/redis_{{item}}.service"
                  state: absent
                with_items: "{{ redis_ports }}"
          
              - name: Recursively remove directory
                file:
                  path: /etc/redis/redis_{{item}}
                  state: absent
                with_items: "{{ redis_ports }}"
          
              - name: Remove directory
                file:
                  path: /etc/redis/
                  state: absent
          
          • upgrade.yml
            ---
            - include_tasks: installredis.yml
          
        • templates

          • config.j2
            dir /etc/redis/redis_{{redis_port}}
            bind {{redis_ip}} 127.0.0.1
            requirepass {{redispass}}
            masterauth {{redispass}}
            port {{redis_port}}
            pidfile /var/run/redis_{{redis_port}}.pid
            maxclients 100000
            # 啟用 redis cluster
            cluster-enabled yes
            # 每個 node 需要獨立,cluster 自行維護使用,不需人為介入
            cluster-config-file nodes_{{redis_port}}.conf
            # node 判斷失效的時間
            cluster-node-timeout 5000
            maxmemory-policy volatile-lru
            loglevel notice
          
          • service.j2
            [Unit]
            Description=Redis persistent key-value database
            After=network.target
            After=network-online.target
            Wants=network-online.target
          
            [Service]
            ExecStart=/usr/bin/redis-server /etc/redis/redis_{{redis_port}}.conf --supervised systemd
          
            ExecStop=/usr/libexec/redis-shutdown redis/redis_{{redis_port}}
            Type=notify
            User=redis
            Group=redis
            RuntimeDirectory=redis_{{redis_port}}
            RuntimeDirectoryMode=0755
            LimitNOFILE=10000000
            LimitCORE=infinity
            LimitNPROC=infinity
            LimitMEMLOCK=infinity
          
            [Install]
            WantedBy=multi-user.target
          
        • vars

          • main.yml
            redispass: pass.123
            redisversion: redis-6.0.6-1.el7.remi
          
        • inventories

          • dev.ini
            [redis_all]
            redis1 ansible_host=10.0.1.5 redis_ip=10.0.1.5 ansible_ssh_user=root ansible_ssh_pass=pass.123 redis_ports=7000,7003
            redis2 ansible_host=10.0.1.6 redis_ip=10.0.1.6  ansible_ssh_user=root ansible_ssh_pass=pass.123 redis_ports=7001,7004
            redis3 ansible_host=10.0.1.7 redis_ip=10.0.1.7 ansible_ssh_user=root ansible_ssh_pass=pass.123 redis_ports=7002,7005
          
        • README.md

          ## 全新安裝
        
              ```bash
              ansible-playbook -i roles/redis-replication/inventories/dev.ini -e "purpose=install" main.yml
              ```
        
          ## 重新安裝
        
              > 先 uninstall 並排除安裝基本套件
        
              ```bash
              ansible-playbook -i roles/redis-replication/inventories/dev.ini -e "purpose=reinstall" main.yml
              ```
        
          ## 更新 config
        
              ```bash
              ansible-playbook -i roles/redis-replication/inventories/dev.ini main.yml
              ```
        
          ## 升級
        
              ```
              ansible-playbook -i roles/redis-replication/inventories/dev.ini -e "purpose=upgrade" main.yml
              ```
          ## restart service
        
              ```
              ansible-playbook -i roles/redis-replication/inventories/dev.ini -e "purpose=restart" main.yml
              ```
        
    • ansible.cfg

      [defaults]
      host_key_checking = False
      roles_path = ./roles
      
    • main.yml

      - hosts: redis_all
        roles:
          - redis-cluster
    

心得

完整程式碼請參考 ansible-redis-cluster

此次修改重點如下:

  1. Redis 版本從 5.0.7 升級為 6.0.6
  2. 控制流程變數由 action 改用 purpose (action 是保留字)
  3. defaults 放變數預設值;vars 放常數值
  4. redis_portredis_port 可以用來處理單台 host 多個 redis instance,效能較好

參考資訊

  1. Ansible 安裝 Redis Cluster
  2. ansible-redis-cluster