Creating a Watch Folder

Important: If you upgraded from Enterprise Server 3.6.1, Connect Server 3.6.1, or Point-to-Point 3.6.1, and you had the asperawatchd and asperawatchfolderd services enabled and configured, you may have to make manually migrate your services depending on the user is running the services. For further details, see Migrating Watch Services on Upgrade.

A watch folder is an automation of file transfers from a source to a destination system. Files placed into a source folder are automatically transferred to the destination. It runs on the client side only and the recipient Aspera server endpoint does not need additional software components to support receiving data. The instructions below uses curl commands to create watch folders.

Process Overview:
  1. Make sure the asperarund service is running.
    Although asperarund starts automatically upon install, make sure the service is running, as watchfolder uses the service to start the asperawatchd and asperawatchfolderd watch services.
  2. Configure the asperawatchd and asperawatchfolderd watch services.
  3. Create a watch folder using the aswatchfolderadmin utility.
You can also configure watch folders by interacting with the Node API or by using IBM Aspera Console. For more information, see Creating a Watch Folder with the Node API and the IBM Aspera Console Admin Guide.

Configuring Watchfolder Services

Watchfolders uses the Aspera Watchfolder service (asperawatchfolderd) and the output from the Aspera Watch Service (asperawatchd) to automatically transfer new files added to a source folder on an Aspera node. Follow the instructions below to start the two services.

  1. Select a user account to run your services.

    Watchfolder services must be run under a user with access to every area of your file system in which you intend to create a watch folder. Watchfolders allows you to run multiple instances of these services under different users. Most users run these services under one user. Choose a user that has access to your entire file system.

    If you need to run multiple instances of these services to access every area of your file system, see Choosing User Accounts to Run Watchfolder Services.

  2. Configure a docroot for the chosen user.
  3. Ensure the user has permissions to write to the default log directory if no directory is specified.
    For more information about configuring log directories, see asperawatchd Configuration.
  4. Configure the asperawatchd and asperawatchfolderd services to run under your user.

    The following commands add the services in the asperarund database; asperarund automatically starts and preserves services in its database. The asperarund service must be running.

    # /opt/aspera/sbin/asperawatchd --user username
    # /opt/aspera/sbin/asperawatchfolderd --user username
  5. Verify that asperawatchd and asperawatchfolderd is running under the given user.

    Use the aswatchadmin and aswatchfolderadmin admin tools to retrieve a list of running daemons. Daemons are named with the username you passed in when starting the service. For example, if you used the root user to run your services, you should see the root daemon listed when you run the following commands:

    # /opt/aspera/bin/aswatchadmin query-daemons
    [aswatchadmin query-daemons] Found a single daemon:
    	root
    
    # /opt/aspera/bin/aswatchfolderadmin query-daemons
    [aswatchfolderadmin query-daemons] Found a single daemon:
    	root

Creating a Watch Folder with the aswatchfolderadmin Utility

The instructions below uses the aswatchfolderadmin utility to create watch folders.
  1. Create and configure a JSON configuration file for your watch folder.

    You must configure the following fields for a valid JSON configuration for a watch folder:

    Field Description
    id (optional) A unique ID for the watchfolder. If this field is not configured, a uuid is automatically generated for the watchfolder.
    source_dir The source directory, relative to the user's docroot. When asperawatchd detects a new file in this location, asperawatchfolderd starts an ascp session to transfer the file to target directory. This directory must be within the docroot set for the user running asperawatchd.
    target_dir The target directory, relative to the user's docroot. If this is a remote directory, define the remote machine in the host field and provide SSH crednetials for the remote machine in the user and pass fields.
    host The host of the target directory. The host can be specified with an IPv4 or IPv6 address.

    The preferred format for IPv6 addresses is x:x:x:x:x:x:x:x, where each of the eight x is a hexadecimal number of up to 4 hex digits. Zone IDs (for example, %eth0) can be appended to the IPv6 address.

    user The username for the host of the target directory.
    pass The password for the host of the target directory.
    {
        "source_dir": "source_directory",
        "target_dir": "target_directory",
        "transport": {
            "host": "host",
            "user": "username",
            "pass": "password"
        }
    }
    For example:
    {
        "source_dir": "/src",
        "target_dir": "/dest",
        "transport": {
            "host": "198.51.100.22",
            "user": "watchfolder_admin",
            "pass": "XF324cd28"
        }
    }

    For a full reference to the options you can configure, see Configuring the Watch Folder JSON Configuration File.

    Save the configuration file.

  2. Create a watch folder using the aswatchfolderadmin utility.

    The aswatchfolderadmin utility creates a watch folder given the ID of an asperawatchfolderd daemon and a properly formatted JSON file. You can use the aswatchfolderadmin query-daemons command to retrieve a list of daemons.

    To create a new watch folder, run the following command:

    # /opt/aspera/bin/aswatchfolderadmin create-folder daemon_name -f json_file
    For example, given the root daemon and a valid JSON file saved as watchfolderconf.json, the output of the aswatchfolderadmin command should look like the following:
    # /opt/aspera/bin/aswatchfolderadmin create-folder root -f watchfolder_conf.json
                                [aswatchfolderadmin create-folder] Successfully created instance b394d0ee-1cda-4f0d-b785-efdc6496c585.
    If aswatchfolderadmin errors out with error code err=28672, check that your docroot has been properly configured to provide access to the source directory. If you need to make changes to your docroot, see Updating the Docroot of a Running asperawatchfolderd Service.
  3. Verify that the watch folder is running.

    Use the aswatchfolderadmin utility to retrieve a list of running watch folders:

    # /opt/aspera/bin/aswatchfolderadmin query-folders daemon_name
    For example:
    # /opt/aspera/bin/aswatchfolderadmin query-folders root
                                [aswatchfolderadmin query-folders] Found a single watchfolder:
                                b394d0ee-1cda-4f0d-b785-efdc6496c585
  4. Test your watch folder.
    Add files to the source directory you configured in the JSON configuration file. If the configuration is correct, Watchfolder detects the new files, starts a transfer, and the added files appear in the target directory.
After creating your watch folder, you can manage the watch folder using the aswatchfolderadmin utility. For more information, see Managing Watch Folders with aswatchfolderadmin.