Transferring and Deleting Files with the Aspera Watch Service

When used in conjunction with ascp commands, the Aspera Watch Service (asperawatchd) allows for fast detection and sending of new and deleted items. By comparing snapshots of the file directory it is watching, asperawatchd generates file lists for ascp commands for transfers requests. To generate snapshots and file lists, you must first configure and start asperawatchd. For more information, see Configuring the Aspera Watch Service.

Process Overview

1. Determine the daemon under which the watchd service is running.
2. Retrieve the uuid of the watcher for the watched directory.
3. Take a snapshot of the current state of the directory.
4. Generate a snapshot differential between the previous state and the current state and save it in a file list.
5. Use the file list in an ascp command (to send created and modified files) or in an asdelete command (to delete files).

Viewing Active watchd Daemons

To view daemons under which watchd service is running, run the following command:

# /opt/aspera/bin/aswatchadmin query-daemons 

The output will be similar to the following example in which root is the daemon:

Found 2 daemons: root

Retrieving the Watcher uuid

To generate a file list for all the changes in a directory, first retrieve the uuid of the watcher of the watched directory.

# /opt/aspera/bin/aswatchadmin query-watches domain

The output will have the following format:

Found a single watch: uuid, path=filepath

The uuid looks like e565a3e3-e25e-44c2-9d68-db1d59cef4c7. Copy the uuid to input into the snapshot command.

Taking a Snapshot

A snapshot represents the structure of the file system at a particular point of time; asperawatchd compares two snapshots to detect file system changes and generate a file list of those changes. To take a snapshot, run the following command with the watcher uuid associated with the file system.

# /opt/aspera/bin/aswatchadmin create-snapshot domain uuid

For example, to take a snapshot of the watch with the uuid from the previous section:

# /opt/aspera/bin/aswatchadmin create-snapshot root e565a3e3-e25e-44c2-9d68-db1d59cef4c7
Successfully created snapshot 1.

Sending New and Modified Files

Take another snapshot of the directory, generate and save a list of the files that have been created or modified since the previous snapshot, then run an ascp command to send those files to another location.

# /opt/aspera/bin/aswatchadmin snapshot-differential domain uuid snapshot_number_1 snapshot_number_2 --exclude-remove --escape + --format +PATH
# ascp --file-list /filelist --source-prefix=/prefix --mode=send --user=username --host=host /target_directory

For example:

# /opt/aspera/bin/aswatchadmin snapshot-differential root e565a3e3-e25e-44c2-9d68-db1d59cef4c7 1 2 --exclude-remove --escape + --format +PATH
# ascp --file-list ~/filelist --source-prefix=/data/D1 --mode=send --user=aspera --host=10.0.0.1 /data/D2 
file1                                       100%   10MB    9.7Mb/s     00:07    
file2                                       100%   10MB    9.7Mb/s     00:16    
file3                                       100%   10MB    9.7Mb/s     00:26    
file4                                       100%   10MB    9.7Mb/s     00:33    
file5                                       100%   10MB    9.7Mb/s     00:42
file6                                       100%   10MB    9.7Mb/s     00:51
Completed: 61440K bytes transferred in 52 seconds
 (96454 bits/sec), in 6 files.

Removing Files from the Target Directory

The asdelete utility compares the source directory with the target directory and deletes extraneous files from the target directory.

# /opt/aspera/bin/asdelete --host host --auth-name username --auth-pass password /source_directory /target_directory

For example:

# /opt/aspera/bin/asdelete --host 10.0.0.1 --auth-name root --auth-pass !XF345lui@0 /data/D1 /data/R1

Check the target directory to confirm deletion of the correct files.

Other aswatchadmin Commands

To view a list of all aswatchadmin commands, run the following:

# /opt/aspera/bin/aswatchadmin --help