Watch Folder Configuration Reference
When you create or edit a Watch Folder in the GUI, you must specify the Watch Folder service, the source path, Watch scan period, and the remote connection. Other settings can be left with their default values or configured to meet your requirements. The following tables describe available Watch Folder settings by tab.
Watch Folder
These settings configure the Watch Folder source and destination, and the connection to the remote host.
Setting | Description | Default |
---|---|---|
Watch Folder Service | The Watch Folder service, defined by its user and service ID, that the Watch Folder uses. | The first Watch Folder service in the list of services alphabetized by username. |
Watch Folder name | A unique name for the Watch Folder. The value specified in this field is added to the cookie reported by ascp. Optional. | None specified |
Watchd scan period | How frequently the Watch Service scans the source path for file system changes. For
file systems with file notifications (Linux, Windows, macOS), set to 30 minutes (30m) to
provide a backup to the notification system, or set to "infinite" to never scan. On file systems without file notifications, such as
object storage, mounted storage (NFS), Solaris, AIX, and Isilon, file system
scans triggered by the scan period are used to detect file changes. In this
case, set the scan period to frequently scan for changes. On operating systems
that support file notifications (Linux, Windows, macOS), asperawatchd
uses the file notifications as the primary means for detecting changes, and the
scan period serves as a backup. In this case, the default value of 30 minutes
is usually acceptable and no change is necessary. To never scan, and rely
entirely on file notifications, set to For pull Watch Folders, file systems scans that are triggered by scan_period are the sole means for detecting changes in the source directory. Lower scan periods detect changes faster but can result in greater resource consumption, particularly for object storage. |
30m |
Direction | Select Push to transfer from the local computer to a remote server.Select Pull to transfer from the remote server to the local computer. | Push |
Local path | Click Browse to select the local path. The local path is the source for push Watch Folders, and the target for pull Watch Folders. | None specified |
Host | The IP address, DNS, hostname, or URL of the remote server. If you enter the host
manually, use the following syntax based on the type of remote endpoint and authentication
method:
|
None specified |
User | The username for authentication. Required. Depending on the type of authentication, it is the transfer user's username, Node API username, Shares username, or access key ID. For SSH, the system account username. For a Shares server, the Shares username. For a node URL host, the Node API username or access key ID. | None specified |
Authentication | For SSH authentication, select Password and enter the user's
password, or select Public Key and select the user's public key from
the drop-down menu. For Shares hosts, enter the Shares user's password. For node authentication, enter the Node API user's password or access key secret. |
Password |
SSH Port (TCP) | The port to use for SSH connections. | 22 |
FASP port (UDP) | The port to use for UDP connections. | 33001 |
SSH host key fingerprint | The SSH fingerprint of the remote server. Aspera strongly recommends using SSH fingerprint for security. If the fingerprint does not match that of the server, the transfer fails with the error "Remote host is not who we expected". For more information, see Securing Your SSH Server ("Configuring Transfer Server Authentication"). | None specified |
Connection timeout | How long to wait for a connection to respond before failing. | 10s |
Proxy URL | If using, the address of an IBM Aspera Proxy server. The proxy syntax is: dnat(s)://user:password@server:port | None specified |
Remote path | Click Browse to select the source or target directory on the
remote server. The remote path must be within the user's docroot. For access key authentication, the path is relative to the storage specified in the access key. |
None specified |
Settings
These settings define how Watch Folder watches the file system and groups new files added to the source folder into "drops". Drops are groups of files that are transferred together in one session, post-processed together, and reported as a unit.
Setting | Description | Default |
---|---|---|
Sample period | Period used to compute the current bandwidth. Used with queue_threshold to compute the amount of data pushed to ascp. | 2s |
Queue threshold | Watch Folders controls the amount of data pushed to ascp for transferring. When the capacity is reached, Watch Folders waits before pushing new data. This capacity is based on the effective bandwidth reported by ascp. | 5s |
Snapshot creation period | The interval during which Watch Folders groups new files in the source directory into a drop. All files in a drop are transferred in the same transfer session, post-processed together, and reported as a unit. Watch Folders uses asperawatchd to detect file system modifications, and continuously creates snapshots to compute the snapshot differential. A small value results in high temporal resolution for detecting file system modifications, whereas a large value improves asperawatchd performance. Default: 3s. | 3s |
Detection strategy | The strategy that Watch Folders uses to create drops when new files are added
to the source folder:
|
Cool off only |
(Drops) Detection cool off | The time after the first new file is added to the source file during which any other new files are included in the same drop. This setting is only relevant if the detection strategy is Cool off only. Aspera recommends setting the detection cool off to a multiple of the Snapshot creation period for predictable results. | 5s |
Maximum parallel ascp | The maximum number of concurrent ascp sessions that Watch Folders can start. | 10 |
(Files) Detection cool off | How long the Watch Folder service waits for files in the watched folder to stop changing (stabilize) before taking a directory snapshot and creating a drop. Default: 5s. | 3s |
Filters | Restrict the files that are included in the Watch Folder transfer by setting filters.
Click Setting an Include filter protects matching files from subsequent Exclude filters. Filters are applied in order. Watch Folders supports glob and Regex filters. The glob filter system is the same as Ascp; see Using Filters to Include and Exclude Files. Note: An include rule must be followed by
at least one exclude rule, otherwise all files are transferred because none are
excluded. To exclude all files that do not match the include rule, use
/** for glob or .* for Regex. ![]() |
None specified |
Transfer
These settings configure the ascp transfer sessions that transfer files in each drop.
Setting | Description | Default | ||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Bandwidth policy |
|
Fair | ||||||||||||||||||||||||||||||||||||||||
Target rate | The target transfer rate. Transfer at rates up to the specified target rate. This option accepts suffixes T for terabits/s, G for gigabits/s, M for megabits/s, K for kilobits/s, or B for bits/s. Decimals are allowed. If this option is not set by the client, the setting in the server's aspera.conf is used. If a rate cap is set in the local or server aspera.conf, the rate does not exceed the cap. | 10.00 Mbps | ||||||||||||||||||||||||||||||||||||||||
Minimum rate | Attempt to transfer no slower than the specified minimum transfer rate. | 0 bps | ||||||||||||||||||||||||||||||||||||||||
Transport Encryption |
Select the encryption cipher. Aspera
supports three sizes of AES cipher keys (128, 192, and 256 bits) and
supports two encryption modes, cipher feedback mode (CFB) and
Galois/counter mode (GCM). The GCM mode encrypts data faster and
increases transfer speeds compared to the CFB mode, but the server must
support and permit it.
Cipher rules The encryption cipher that you are allowed to use depends on the server configuration and the version of the client and server:
Cipher Values
Client-Server Cipher Negotiation The following table shows which encryption mode is used depending on the server and client versions and settings:
|
AES-128 | ||||||||||||||||||||||||||||||||||||||||
SSH host key fingerprint | The SSH fingerprint of the remote server. Aspera strongly recommends using SSH fingerprint for security. If the fingerprint does not match that of the server, the transfer fails with the error "Remote host is not who we expected". For more information, see Securing Your SSH Server ("Configuring Transfer Server Authentication"). | None specified | ||||||||||||||||||||||||||||||||||||||||
Token | If required, the token string. Not valid for use with growing files. | None specified | ||||||||||||||||||||||||||||||||||||||||
Tags | Specify custom metadata in JSON format. The tags object is passed directly to the ascp session. For more information on writing custom metadata for uploads to object storage (as in the example), see Writing Custom Metadata for Objects in Object Storage. | None specified | ||||||||||||||||||||||||||||||||||||||||
Read block size | The read block size. | None specified (uses the value set in aspera.conf). | ||||||||||||||||||||||||||||||||||||||||
Write block size | The write block size. | None specified (uses the value set in aspera.conf). | ||||||||||||||||||||||||||||||||||||||||
Datagram size | The datagram size (MTU) for FASP. | None specified (uses the detected path MTU). | ||||||||||||||||||||||||||||||||||||||||
Rex message size | The maximum size of a retransmission request. Maximum: 1440. | None specified | ||||||||||||||||||||||||||||||||||||||||
Raw ascp options |
Specify ascp options and their arguments that
are not yet available in Watch Folders to apply to Watch Folder
transfers. To use raw options, they must be enabled in the client's
aspera.conf by running the following
command:
|
None specified | ||||||||||||||||||||||||||||||||||||||||
Cookie | Each transfer session (drop) includes a cookie that provides information about the
Watch Folder that initiates the transfer session and the drop. The cookie has the following
format:
You can override the default values by specifying your own. |
None specified (use default values) | ||||||||||||||||||||||||||||||||||||||||
Content protection | Enter a password to enable client-side encryption
at rest. Files that are uploaded to the server are appended with a
.aspera-env extension. To download and decrypt
.aspera-env files from the server, the client must provide
the password. For more information on client-side encryption at rest, see Client-Side Encryption-at-Rest (EAR).
To disable content protection, clear the check box. |
None specified (content protection disabled) | ||||||||||||||||||||||||||||||||||||||||
Number of retry attempts | How many times to try transferring a file before the file is marked as failed. | 3 | ||||||||||||||||||||||||||||||||||||||||
Wait between retries | How frequently to retry file transfers. | 3s | ||||||||||||||||||||||||||||||||||||||||
Retry maximum for | If no bytes are transferred during the specified period and no file is completed, the drop and all remaining incomplete files in the drop are marked as failed. | 1m |
File Handling
These settings configure how files and their attributes are handled on the source and destination.
Setting | Description | Default |
---|---|---|
Resume policy | Specify if partial files are resumed. To always re-transfer files, select
Never. To resume conditionally, select from the following options:
When resuming file transfer is enabled, select an overwrite rule for when the same file exists at the destination. Click the drop-down menu for If a complete file already exists at the destination. |
Never |
Preserve file UIDs and GIDs | Select to preserve the file owner user ID or group ID. | None selected |
Preserve file timestamps | Select to preserve all file timestamps. Equivalent to enabling Preserve
creation timestamps, Preserve modification timestamps, and
Preserve access timestamps. Note: Access, modification, and source
access times cannot be preserved for node and Shares connections that are using cloud
storage. |
Not enabled |
Preserve creation timestamps | Set creation time of the destination be set to that of the source. If the destination is a non-Windows host, this option is ignored. | Not enabled |
Preserve modification timestamps | Set the modification time of the destination file to that of the source. | Not enabled |
Preserve access timestamps | Set the access time of the destination to that of the source. The destination file has the access time of the source file prior to the transfer. | Not enabled |
Source Handling | Select what to do with source files after they are successfully transferred to the
destination. Files can be
archived, deleted, or retained after transfer of a drop. When files are
archived or deleted, source sub-directories are also deleted from the source,
unless the sub-directories were empty to start. File structure is preserved in
the archive.
File archiving (Automatically move source files to an archive folder) is not supported for sources in object storage.
|
Do nothing after transfer |
Growing Files
Growing files (files that are written to the source folder from a streaming input) are transferred using FASPStream technology rather than ascp. Identify growing files by specifying filters; files that match the filters are considered growing files. This feature requires a growing files-enabled license.
These settings configure how growing files are identified and the transfer session.
Setting | Description | Default |
---|---|---|
Maximum active drops | The maximum number of concurrent FASPStream sessions the Watch Folder can initiate. | 8 |
Bandwidth policy |
|
Fair |
Target rate | The target transfer rate. Transfer at rates up to the specified target rate. This option accepts suffixes T for terabits/s, G for gigabits/s, M for megabits/s, K for kilobits/s, or B for bits/s. Decimals are allowed. If this option is not set by the client, the setting in the server's aspera.conf is used. If a rate cap is set in the local or server aspera.conf, the rate does not exceed the cap. | 10.00 Mbps |
Minimum rate | Attempt to transfer no slower than the specified minimum transfer rate. | 0 bps |
Datagram size | The datagram size (MTU) for FASP. | None specified |
Transport encryption | Select the encryption cipher (AES-128) to use for encrypting data in transit, or disable encryption by selecting none. | AES-128 |
SSH Port (TCP) | The port to use for SSH connections. | 22 |
FASP Port (UDP) | The port to use for UDP connections. | 33001 |
Completion timeout | How long to wait before the session is considered complete. A growing file is considered complete when no new data arrives within the timeout period. | 5s |
Send data after maximum | Force FASPStream to send data after the given time, even if the chunk is not full. | 2s |
Memory | The maximum amount of memory FASPStream is allowed to use. | 2.00 MB |
Chunk size | Packet size for transfers over the network. | 128.00 KB |
Filters | Identify growing files as those that match the specified filters. Click ![]() Note: An include rule must be followed by
at least one exclude rule, otherwise all files are transferred because none are
excluded. To exclude all files that do not match the include rule, use
/** for glob or .* for Regex. Click |
None specified |
Packages
These settings enable the use of package files to define the order of files in the transfer queue for a session, and specify which file, either the last in the list or the package file itself, to transfer last. The package file is identified by setting matching filters.
Setting | Description | Default |
---|---|---|
Package timeout | How long to wait for file dependencies to be satisfied (files that must be transferred before the last file are transferred) before considering the dependency as unsatisfied. | 10s |
Parsers | Click ![]()
Identify package files as those that match the specified filters. Click Note: An include rule must be followed by
at least one exclude rule, otherwise all files are transferred because none are
excluded. To exclude all files that do not match the include rule, use
/** for glob or .* for Regex. Click |
None specified |