Manual Failover with ACM

  1. On the active node, disable ACM locally.
    $ /opt/aspera/acm/bin/acmctl –d
    ACM is disabled locally
  2. Stop all services.
    $ asctl all:stop
    Orchestrator is being stopped forcefully: 
    -> Killing Orchestrator Engine process with pid: 24542 
    -> Killing Mongrel serving orchestrator on port 3000 with pid: 24638 
    -> Killing Mongrel serving orchestrator on port 3001 with pid: 24641 
    -> Killing Mongrel serving orchestrator on port 3002 with pid: 24644 
    -> Orchestrator Monitor running with pid: 24663 
    -> Killing Orchestrator Monitor with pid: 24663 
    -> Killing Asynchronous Worker Process 0 with pid 24665 
    -> Killing Asynchronous Worker Process 1 with pid 24668 
    -> Killing Synchronous Worker Process 2 with pid 24670 
    -> Killing Synchronous Worker Process 3 with pid 24672 
    -> Killing Synchronous Worker Process 4 with pid 24675 
    -> Killing Synchronous Worker Process 5 with pid 24677
    Apache: Stop... done
    MySQL: Stop... done
  3. Check the status of ACM.
    $ /opt/aspera/acm/bin/acmctl –i
    Checking current ACM status...
    Aspera Cluster Manager for Orchestrator - status
    ------------------------------------------
    Local hostname:              orchestrator1
    Active node:                 orchestrator1 (me)
    Status of this node:         active
    Status file:                 current
    Disabled globally:           no
    Disabled on this node:       yes
    
    Database configuration file
    ---------------------------
    Database host:         localhost
    
    Orchestrator active/active services status
    ------------------------------------
    Apache:                stopped
    
    Orchestrator Status:  
      -> Orchestrator Engine NOT running 
      -> Mongrel serving orchestrator on port 3000 is NOT running 
      -> Mongrel serving orchestrator on port 3001 is NOT running 
      -> Mongrel serving orchestrator on port 3002 is NOT running 
      -> Orchestrator Monitor NOT running 
      -> Asynchronous Worker Process 0 is NOT running 
      -> Asynchronous Worker Process 1 is NOT running 
      -> Synchronous Worker Process 2 is NOT running 
      -> Synchronous Worker Process 3 is NOT running 
      -> Synchronous Worker Process 4 is NOT running 
      -> Synchronous Worker Process 5 is NOT running 
    
    Orchestrator active/passive services status
    ------------------------------------------
    MySQL:                 stopped
  4. After a minute or so, confirm that the peer node is active.
    $ /opt/aspera/acm/bin/acmctl -i
    Checking current ACM status...
    Aspera Cluster Manager for Orchestrator - status
    ------------------------------------------
    Local hostname:         orchestrator2
    Active node:            orchestrator2 (me)
    Status of this node:    active
    Status file:            current
    Disabled globally:      no
    Disabled on this node:  no
    
    Database configuration file
    ---------------------------
    Database host:         localhost
    
    Orchestrator active/active services status
    ------------------------------------
    Apache:                 running
    
    Orchestrator Status: 
    -> Orchestrator Engine running with pid: 31524 
    -> Mongrel serving orchestrator on port 3000 is running with pid: 31636 
    -> Mongrel serving orchestrator on port 3001 is running with pid: 31681 
    -> Mongrel serving orchestrator on port 3002 is running with pid: 31685 
    -> Orchestrator Monitor running with pid: 31713 
    -> Asynchronous Worker Process 0 is running with pid: 31715 
    -> Asynchronous Worker Process 1 is running with pid: 31717 
    -> Synchronous Worker Process 2 is running with pid: 31719 
    -> Synchronous Worker Process 3 is running with pid: 31721 
    -> Synchronous Worker Process 4 is running with pid: 31723 
    -> Synchronous Worker Process 5 is running with pid: 31725 
    
    Orchestrator active/passive services status
    ------------------------------------------
    MySQL:                 running
  5. On the newly passive node, enable ACM to automatically start the active services.
    $ /opt/aspera/acm/bin/acmctl –e
    ACM is enabled locally
  6. Check the status on the newly passive node.
    $ /opt/aspera/acm/bin/acmctl -i
    Checking current ACM status...
    
    Aspera Cluster Manager for Orchestrator - status
    ------------------------------------------
    Local hostname:            orchestrator1
    Active node:               orchestrator2
    Status of this node:       passive
    Status file:               current
    Disabled globally:         no
    Disabled on this node:     no
    
    Database configuration file
    ---------------------------
    Database host:         localhost
    
    Orchestrator active/active services status
    ------------------------------------
    Apache:                running
    
    Orchestrator Status: 
      -> Orchestrator Engine running with pid: 26932 
      -> Mongrel serving orchestrator on port 3000 is running with pid: 26967 
      -> Mongrel serving orchestrator on port 3001 is running with pid: 26970 
      -> Mongrel serving orchestrator on port 3002 is running with pid: 26973 
      -> Orchestrator Monitor running with pid: 26979 
      -> Asynchronous Worker Process 0 is running with pid: 26981 
      -> Asynchronous Worker Process 1 is running with pid: 26983 
      -> Synchronous Worker Process 2 is running with pid: 26985 
      -> Synchronous Worker Process 3 is running with pid: 26987 
      -> Synchronous Worker Process 4 is running with pid: 26989 
      -> Synchronous Worker Process 5 is running with pid: 26991
    
    Orchestrator active/passive services status
    ------------------------------------------
    MySQL:                 running