Getting Started with Watch Folders in the Command Line

Watch Folders enables large-scale, automated file and directory transfers, including ultra-large directories with over 10 million items and directories with "growing" files. Watch Folders use input from asperawatchd to automate transfers of files added to or modified in a source folder. They can be configured to push from the local server or pull from a remote server. Remote servers can be HST Server, HST Endpoint, and IBM Aspera Shares servers, as well as servers in object storage. Push Watch Folders can use IBM Aspera on Cloud and IBM Aspera Transfer Cluster Manager nodes for a destination.

1. Ensure that asperarund is running.
   Run the following command:

   # systemctl status asperarund
   Aspera Run Server: asperarund           [ RUNNING ]

   Or for Linux systems that use init.d:

   # service asperarund status
   Aspera Run Server: asperarund           [ RUNNING ]

2. Select or create a user account to run your services.
   Watch Folder services must be run under a user with access to every area of your file system in which you intend to create a Watch Folder. You can run multiple instances of these services under different users; however, most deployments run these services under one user. Choose a user that has access to your entire file system.

   If you need to run multiple instances of these services to access every area of your file system, see Choosing User Accounts to Run Watch Folder Services.

3. Configure a docroot or restriction for the user.
   Docroots and path restrictions limit the area of a file system or object storage to which the user has access. Users can create Watch Folders and Watch services on files or objects only within their docroot or restriction.

   Note: Users can have a docroot or restriction, but not both or Watch Folder creation fails.

   Docroots can be set up in the GUI or command line. In the GUI, click Configuration > Users > username > Docroot and set the permitted path as the value for Absolute Path. To set up a docroot from the command line, run the following command:

   # asconfigurator -x "set_user_data;user_name,username;absolute,docroot"

   Restrictions must be set from the command line:

   # asconfigurator -x "set_user_data;user_name,username;file_restriction,|path"

   The restriction path format depends on the type of storage. In the following examples, the restriction allows access to the entire storage; specify a bucket or path to limit access.

   Storage Type	Format Example
   local storage	For Unix-like OS: specific folder: file:////folder/* drive root: file:////* For Windows OS: specific folder: file:///c%3A/folder/* drive root: file:///c*
   Amazon S3 and IBM Cloud Object Storage - S3	s3://*
   Azure	azu://*
   Azure Files	azure-files://*
   Azure Data Lake Storage	adl://*
   Alibaba Cloud	oss://*
   Google Cloud	gs://*
   HDFS	hdfs://*

   With a docroot or restriction set up, the user is now an Aspera transfer user. Restart asperanoded to activate your change:

   Run the following commands to restart asperanoded:

   # systemctl restart asperanoded

   or for Linux systems that use init.d:

   # service asperanoded restart

4. Ensure the user has permissions to write to the default log directory if no directory is specified.
   For more information about configuring log directories, seeWatch Service Configuration.
5. Configure asperawatchd and asperawatchfolderd settings.
   Though the default values are already optimized for most users, you can also configure the snapshot database, snapshot frequency, logging, scan threads, and drop handling, among other features. For instructions, see Watch Service Configuration and Watch Folder Service Configuration.
6. Configure Linux for many Watch Folders.
   If you plan to watch more than 8,200 directories on a Linux computer, you might need to configure it to support that many processes. For instructions, see Configuring Linux for Many Watch Folders.