Starting the Aspera Watch Service

The Aspera Watch Service (asperawatchd) is a file system change detection and snapshot service for speed, scale and distributed sources. It discovers changes (such as new files and directories, deleted items, and renames) in source file systems as they occur, eliminating the need to scan the file system. It can be used on any local or shares (CIFS, NFS) host.

When used in conjunction with ascp commands, the Aspera Watch Service allows for fast detection and send of new and deleted items. By comparing snapshots of the file directory it is watching, asperawatchd generates file lists for ascp commands for transfers and delete requests.

Aspera Sync can be configured to use asperawatchd for fast synchronization of very large numbers of files without scanning the directory. Aspera Sync can push files from a local directory, pull files from a remote directory, or create a bi-directional session between two directories (as long as asperawatchd is properly configured to monitor both directories). For more information on using Sync, see the IBM Aspera Sync User Guide.

To push files to a remote host using Aspera Watch Service, configure asperawatchd on the local host. The remote host does not need to be configured. To pull files from a remote host, configure asperawatchd on the remote host. The local host does not need to be configured.

To start asperawatchd, follow these steps:

  1. Configure a docroot for the user who will run the service.

    The docroot is a security feature that allows you to restrict the area asperawatchfolderd can access. If you need to acces the entire file system, you can set the docroot path as /, but you cannot leave it empty.

    Set the docroot for the user using the asconfigurator utility:

    # asconfigurator -x "set_user_data;user_name,username;absolute,docroot"

    For example:

    # asconfigurator -x "set_user_data;user_name,root;absolute,/"
    success
    user_name: root
    
    
    The asconfigurator command adds the following configuration to the <aaa> section of aspera.conf:
    <aaa>
        <realms>
            <realm>
                <users>
                    <user>
                        <name>root</name>
                        <file_system>
                            <access>
                                <paths>
                                    <path>
                                        <absolute>/</absolute>
                                    </path>
                                </paths>
                            </access>
                        </file_system>
                    </user>
                </users>
            </realm>
        </realms>
    </aaa>
  2. Ensure the user has permissions to write to the default log directory if no directory is specified.
  3. To pull files from a remote host, on the remote host set the data storage for either default or for specific users.

    Setting data storage under default uses asperawatchd for all pull requests to the server whereas setting data storage for a specific user means that asperawatchd is used only for pull requests by that user. You can use asconfigurator to set either of these, or you can edit aspera.conf manually. To add the data storage to the default section, run the following command:

    #asconfigurator -x "set_node_data;async_watchd,redis:hostname:31415[:domain]"

    This adds the following line to aspera.conf:

    <default>
        <async_watchd>redis:hostname:31415:domain</async_watchd>
    </default>

    To add the data storage to a specific user, run the following command:

    #asconfigurator -x "set_user_data;user_name,username;async_watchd,redis:hostname:31415[:domain]"

    This adds the following line to aspera.conf:

    <aaa>
        <realms><realm>
            <users>
                <user>
                    <name>username</name>
                    <file_system>
                        <access><paths><path>
                            <absolute>/data</absolute>
                            </path></paths></access>
                    </file_system>
                    <async_watchd>redis:hostname:31415[:domain]</async_watchd>
                </user>
            </users>
        </realm></realms>
    </aaa>
    Validate the aspera.conf file using the asuserdata utility:
    # /opt/aspera/bin/asuserdata -v
  4. Configure the asperawatchd service to run under your user.
    The following command adds the asperawatchd service to the asperarund database; asperarund automatically starts and preserves services in its database. The asperarund service must be running.
    # /opt/aspera/sbin/asperawatchd --user username
  5. Verify that asperawatchd is running under the given user.

    Use the aswatchadmin utlity to retrieve a list of running daemons. Daemons are named with the username you passed in when starting the service. For example, if you used the root user to run your services, you should see the root daemon listed when you run the following commands:

    # /opt/aspera/bin/aswatchadmin query-daemons
    [aswatchadmin query-daemons] Found a single daemon:
    	root
  6. Create a watch.

    A watch is a directory watched by the asperawatchd service. To create a watch, run the following command, where daemon is the username used to start the asperawatchd service and filepath is the directory to watch.:

    # /opt/aspera/bin/aswatchadmin create-watch daemon filepath

    You can configure asperawatchd to watch multiple directories by creating more than one watch, but Aspera recommends specifying a single watch at the top level directory. Though you can specify as many watches as needed, (for example, one for each directory), specifying a single watch is the most efficient use of memory. For example, to monitor both the /data/D1 and /data/D2 directories, specify /data as the path for the watch.

With the Watch Service configured, you can now monitor watches and manage their files as they change.