Setup and Configuration

To set up a highly-available HST Server cluster, you configure HST Server, the Redis database, Redis Sentinel, HAProxy, and syslog on each node. Select one node as the Redis "primary" node and the others are Redis "replica" nodes.

1. On each node, run the following commands to configure HST Server to connect the Redis Client to HAProxy:

   $ asconfigurator -x "set_server_data;db_type,ha_redis"
   $ asconfigurator -x "set_server_data;db_host,127.0.0.1"
   $ asconfigurator -x "set_server_data;db_port,31414"

   Where 31414 is the port on which HAProxy listens.

2. On the Redis primary node, create (or edit an existing) /opt/aspera/etc/redis.conf with the following values:

   bind private_ip_address
   port 31415
   daemonize yes
   pidfile /opt/aspera/var/run/redis.31415.pid
   logfile /opt/aspera/var/run/redis_31415.log
   tcp-keepalive 300
   dbfilename redis.31415.rdb
   dir /opt/aspera/var
   save 900 1
   protected-mode no   

   Where private_ip_address is the private IP address of the node.

   Note: If you want Redis to use the system logger, comment out the logfile /opt/aspera/var/run/redis_31415.log and set syslog-enabled to yes. For example:

   #/opt/aspera/var/run/redis_31415.log
   syslog-enabled yes

   Configure other syslog parameters, if desired.

   Save your changes and close the file.

3. On the Redis replica nodes, create (or edit an existing) /opt/aspera/etc/redis.conf with the following values:

   slaveof primary_ip_address 31415
   bind private_ip_address
   port 31415
   daemonize yes
   pidfile /opt/aspera/var/run/redis.31415.pid
   logfile /opt/aspera/var/run/redis_31415.log
   tcp-keepalive 300
   dbfilename redis.31415.rdb
   dir /opt/aspera/var
   save 900 1
   protected-mode no   

   Where:

   • primary_ip_address is the private IP address of the primary node
   • private_ip_address is the private IP address of the local replica node

   Configure logging as you did on the primary node.

   Save your changes and close the file.

4. On each node, configure Redis Sentinel.
   1. Open port 26379 in iptables.
   2. Open /opt/aspera/etc/sentinel.conf and add or edit the following lines:

      port 26379
      bind private_ip_address
      sentinel monitor primary_name primary_ip_address 31415 quorum
      sentinel down-after-milliseconds primary_name down_time
      sentinel failover-timeout primary_name timeout
      logfile "/opt/aspera/etc/redis-sentinel.log"

      Where:

      • private_ip_address is the private IP address of the local node.
      • primary_name is a name of the primary node. The name can include A-z, 0-9, and ".", "-", or "_"; the name cannot include special characters or spaces.
      • primary_ip_address is the private IP address of the primary node. On the primary node, private_ip_address and primary_ip_address are the same.
      • quorum is the number of Redis Sentinels that must agree that the primary is not reachable.
      • down_time is the number of milliseconds after which a node is reported as failing.
      • timeout is the number of seconds after which a failover process is started.

      Redis Sentinel does not support logging to syslog; it must use a separate file.

      For example:

      port 26379
      bind 10.0.213.131
      sentinel monitor myprimary 10.0.114.111 31415 2
      sentinel down-after-milliseconds myprimary 300
      sentinel failover-timeout myprimary 1000
      logfile "/opt/aspera/etc/redis-sentinel.log"

      Save your changes and close the file.

5. On each node, configure HAProxy.
   1. Open /opt/aspera/etc/haproxy/haproxy.cfg.template:

      ...
      #   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
      # log is set for syslog configuration https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#log
      global
        log          127.0.0.1 local2 info
        chroot      /var/lib/haproxy
        pidfile     /var/run/haproxy.pid
        maxconn     4000
        user        haproxy
        group       haproxy
        daemon
      
      # Specifies TCP timeout on connect for use by the frontend ft_redis
      # Set the max time to wait for a connection attempt to a server to succeed 
      # The server and client side expected to acknowledge or send data.
      defaults REDIS
        mode tcp
        option tcplog  
        timeout connect 3s
        timeout server 4s
        timeout client 4s
      
      # Specifies listening socket for accepting client connections using the default 
      # REDIS TCP timeout and backend bk_redis TCP health check.
      frontend ft_redis
        bind *:31414 name redisft
        default_backend bk_redis
      
      # Specifies the backend Redis proxy server TCP health settings 
      # Ensure it only forward incoming connections to reach a master.
      backend bk_redis
        option tcp-check
        tcp-check comment PING\ phase
        tcp-check send PING\r\n
        tcp-check expect string +PONG
        tcp-check comment role\ check
        tcp-check send info\ replication\r\n
        tcp-check expect string role:master
        tcp-check comment QUIT\ phase
        tcp-check send QUIT\r\n
        tcp-check expect string +OK
        server redis-1 primary_ip_address:31415 maxconn 1024 check inter 1s
        server redis-2 replica1_ip_address:31415 maxconn 1024 check inter 1s
        server redis-3 replica2_ip_address:31415 maxconn 1024 check inter 1s

   2. Edit the last lines in the file, which specify the private IP addresses of the cluster nodes.
      The template shows the configuration for three cluster nodes; add similar lines for additional nodes.
   3. Save the file as /opt/aspera/etc/haproxy/haproxy.cfg and close it.
   4. Verify the configuration:

      $ /opt/aspera/sbin/haproxy -f /opt/aspera/etc/haproxy/haproxy.cfg -c

   5. Configure iptables to ACCEPT the Redis IP addresses.