Getting Started with Watch Folders in the Command Line

The Aspera Watch Folder service (asperawatchfolderd) enables large-scale, automated file and directory transfers, including ultra-large directories with over 10 million items and directories with "growing" files. Watch Folders use input from the Aspera Watch Service to automate transfers of files added to or modified in a source folder. They can be configured to push from the local server or pull from a remote server. Remote servers can be High-Speed Transfer Server, High-Speed Transfer Endpoint, and IBM Aspera Shares servers, as well as servers in object storage. Push Watch Folders can use Aspera on Cloud and Aspera Transfer Cluster nodes for a destination.

Your Aspera product requires configuration to support Watch Folders. Whether you create Watch Folders using the command line tool aswatchfolderadmin (Creating a Push Watch Folder with aswatchfolderadmin) or the Watch Folder API (Creating a Push Watch Folder with the API), prepare your computer by taking the following steps.

  1. Ensure that the asperarund service is running.
    Run the following command:
    # systemctl status asperarund
Aspera Run Server: asperarund           [ RUNNING ]
    Or for Linux systems that use init.d: 
    # service asperarund status
Aspera Run Server: asperarund           [ RUNNING ]
  2. Select or create a user account to run your services.
    Watch Folder services must be run under a user with access to every area of your file system in which you intend to create a Watch Folder. You can run multiple instances of these services under different users; however, most deployments run these services under one user. Choose a user that has access to your entire file system.

    If you need to run multiple instances of these services to access every area of your file system, see Choosing User Accounts to Run Watch Folder Services.

  3. Configure a docroot or restriction for the user.
    Docroots and path restrictions limit the area of a file system or object storage to which the user has access. Users can create Watch Folders and Watch services on files or objects only within their docroot or restriction.
    Note: Users can have a docroot or restriction, but not both or Watch Folder creation fails.

    Docroots can be set up in the GUI or command line. In the GUI, click Configuration > Users > username > Docroot and set the permitted path as the value for Absolute Path. To set up a docroot from the command line, run the following command:

    # asconfigurator -x "set_user_data;user_name,username;absolute,docroot"

    Restrictions must be set from the command line:

    # asconfigurator -x "set_user_data;user_name,username;file_restriction,|path"

    The restriction path format depends on the type of storage. In the following examples, the restriction allows access to the entire storage; specify a bucket or path to limit access.

    Storage Type Format Example
    local storage For Unix-like OS:
    • specific folder: file:////folder/*
    • drive root: file:////*
    For Windows OS:
    • specific folder: file:///c%3A/folder/*
    • drive root: file:///c*
    Amazon S3 and IBM Cloud Object Storage - S3 s3://*
    Azure azu://*
    Azure Files azure-files://*
    Azure Data Lake Storage adl://*
    Swift swift://*
    Alibaba Cloud oss://*
    Google Cloud gs://*
    HDFS hdfs://*

    With a docroot or restriction set up, the user is now an Aspera transfer user. Restart the Aspera NodeD service to activate your change:

    Run the following commands to restart asperanoded:
    # systemctl restart asperanoded
    or for Linux systems that use init.d:
    # service asperanoded restart
  4. Ensure the user has permissions to write to the default log directory if no directory is specified.
    For more information about configuring log directories, seeWatch Service Configuration.
  5. Configure the Aspera Watch Service and Watch Folder service settings.
    Though the default values are already optimized for most users, you can also configure the snapshot database, snapshot frequency, logging, and drop handling, among other features. For instructions, see Watch Service Configuration and Watch Folder Service Configuration.
  6. Configure Linux for many Watch Folders.
    If you plan to watch more than 8,200 directories on a Linux computer, you might need to configure it to support that many processes. For instructions, see Configuring Linux for Many Watch Folders.
Your system is now ready for Watch Folders.

To create a push Watch Folder, see Creating a Push Watch Folder with aswatchfolderadmin or Creating a Push Watch Folder with the API.

To create a pull Watch Folder, see Creating a Pull Watch Folder with aswatchfolderadmin or Creating a Pull Watch Folder with the API.