Configuring the Aspera Watch Service

The Aspera Watch Service (asperawatchd) is a file system change detection and snapshot service for speed, scale and distributed sources. It discovers changes (new files and directories, deleted items, renames, and so on) in source file systems immediately as they occur, eliminating the need to scan the file system. It can be used on any local or shares (CIFS, NFS) host.

When used in conjunction with ascp commands, the Aspera Watch Service allows for fast detection and send of new and deleted items. By comparing snapshots of the file directory it is watching, asperawatchd generates file lists for ascp commands for transfers and delete requests.

  1. To configure asperawatchd, open the aspera.conf file with an editor located at /opt/aspera/etc/aspera.conf and add the <watchd> section.
    • Set <enabled> to true.
    • Set <master> to true.
    • Set a directory for <log_level>.
    • Set the data storage for <db_spec> (datastore:host:port). By default, asperawatchd uses Redis with port 31415.
    • Set directories in <watches> for asperawatchd to monitor.
    <server>
        <watchd>
            <enabled>true</enabled>
            <log_level>log</log_level>
            <log_directory>/tmp</log_directory>
            <max_directories>1000000</max_directories> 
            <max_queue>100000</max_queue>
            <max_snapshots>10000</max_snapshots>
            <master>true</master>
            <db_spec>redis:localhost:31415</db_spec>
            <scan_period>30m</scan_period>
            <watches>
                <watch>
                    <path>/data/D1</path>
                </watch>
            </watches>
        </watchd>
    </server>
    The <watch> section specifies directories for asperawatchd to monitor. When starting a session with Aspera Sync, Sync requests snapshots from asperawatchd, a process which takes longer depending on the number of files in the directory. The asperawatchd service periodically scans the directories specified in aspera.conf, reducing time spent waiting for a new completed scan.
    Tip: You can configure asperawatchd to watch multiple directories, but Aspera recommends specifying a single <watch> at the top level directory. Though you can specify as many watches as needed, (for example, one for each directory), specifying a single watch is the most efficient use of memory. To monitor both the /data and /data/D1 directories, specify /data as the path for the <watch>.
  2. After properly configuring the asperawatchd service, you must manually start the service.
    # service asperawatchd start
    Check to make sure the asperawatchd service is running.
    # ps aux | grep watchd
    root      2306  0.1  0.5 732968 11220 ?        Ssl  Nov13   1:17 /opt/aspera/sbin/asperawatchd