Aspera Sync Command Reference

Syntax

> async [instance_options] -N pair -d ldir -r [user@host:rdir] [session_options] ...
Note: Transfers started by async can be controlled from the Enterprise Server or Connect Server GUI. Canceling an async transfer in a GUI shuts down async.

Required Command Options

Naming the async session: -N pair

-N pair is required in async commands. The value for pair is a name that uniquely identifies the Sync session and is visible in Aspera Console. -N pair must follow any instance options and must precede all session arguments. Names can only use standard alphanumeric characters, plus "_" and "-" characters.

Specifying filepaths and filenames: ldir and rdir

ldir specifies the local directory to be synchronized and rdir specifies the remote directory to be synchronized. File paths and filenames must follow these rules:

Specifying the direction of the sync: -K direction

Sync has three modes of synchronization: push, pull, and bidi.

Using continuous mode: -C

Continuous mode is supported only when the file source is Windows or Linux. See the table below for the operating system requirements for the Sync server and client for the different Sync directions.

Continuous Sync Direction Supported Sync Client OS Supported Sync Server OS
PUSH Linux, Windows All
PULL All Linux, Windows
BIDI Linux, Windows Linux, Windows

One-time synchronization is supported between all operating systems.

The following tables are complete command-line options references. View an abbreviated version from the command line by running:

> async -h

For examples of async commands and output, see Examples of Sync Commands and Output.

Environment Variables

If needed, you can set the following environment variables for use with async:

ASPERA_SCP_COOKIE=cookie
Set the transfer user cookie. Overridden by --cookie.
ASPERA_SCP_LICENSE=license_string
Set a base64-encoded Aspera license string.
ASPERA_SCP_PASS=password
Set the transfer user password. Overridden by -w and --pass.
ASPERA_SCP_TOKEN=token
Set the transfer user authorization token. Overridden by -W and --token.

Instance Options

-A, --version
Display the async version information and license information.
--apply-local-docroot
Prepend the local docroot to the local directory.
-D[D..]
Log at the specified debug level. Default is 0. Additional Ds return more messages.
-h, --help
Display help for command-line options.
-L log_dir, --alt-logdir=log_dir
Log to the specified logging directory on the local host. If the directory doesn't exist, async creates it for you.
-q, --quiet
Disable all output.
--watchd=datastore:host:port[:domain]
Use asperawatchd connected to the specified Redis for the transfer session. datastore can be redis or scalekv.
For example:
--watchd=redis:localhost:31415

The optional domain argument allows you to specify if the domain is other than the default svcAspera. For more information see Working with the Aspera Watch Service.

Session Options

-a policy, --rate-policy=policy
Transfer with the specified rate policy. policy can be fixed, fair, high, or low. Default: fair
--assume-no-mods
Assume that the directory structure has not been modified. If a directory's modification time has not changed compared to the Sync database, async in non-continuous mode skips scanning the directory. This option makes scanning static directory structures faster. Aspera recommends using --exclude-dirs-older-than instead of this option.
-B rdbdir, --remote-db-dir=rdbdir
Save the remote database to the specified directory. Similar to -b, but applies to the remote database. For further usage information, see The Sync Database. Default: ~private-asp at the root level of the synchronized directory. The directory is created if it does not already exist. If <async_db_dir> is set in aspera.conf on the server, that setting overrides the location specified with -B .
-b ldbdir, --local-db-dir=ldbdir
Use the specified local database directory. Default: ~private-asp at the root level of the synchronized directory.

You can save the Sync database to a different location than the default one under the ldir specified with -d. This allows you to store the database away from the main data files, which is useful for performance tuning. It is also useful when -d ldir is located on a network share volume that does not reliably support database locking. For further usage information, see The Sync Database.

-C, --continuous
Run continuous synchronization. Default: disabled.
Note: Continuous mode is supported only when the file source is Windows or Linux. Continuous pulls can be run from any operating system if the source is Windows or Linux. Continuous push can be run only by Windows or Linux. Continuous bidi requires that both the Sync client and server are Windows or Linux.

If you receive an inotify error when attempting to run continuous synchronization, see Troubleshooting Continuous Mode Errors.

If a file is open, async cannot transfer the file due to sharing violations and may ignore the file if it is closed without changes. To specify the maximum number of retries after a sharing violation, use with --sharing-retry-max. To enable periodic scans that detect when an opened file has been closed and is ready for transfer, use with --scan-interval.

-c cipher, --cipher=cipher
Encrypt file data with encryption algorithm. cipher can be none, AES128, AES192, or AES256. Default: AES128.
--check-sshfp=fingerprint
Compare fingerprint to the remote host key hash and fail on mismatch.
--compression={zlib|none}
Compress a file before transfer using the specified method. Default: none.
--cookie=cookie
Specify a user-defined identification string to report to the Aspera Management interface. cookie cannot contain the special characters \r, \n, or \0.
--cooloff=sec
Delay the start of the transfer. For example, if --cooloff=5, async waits 5 seconds before copying a file. If --cooloff=0 transfers start immediately. The client and server use the same cooloff period. Valid range for sec: integers 0-60. Default: 3.
--cooloff-max=sec
Wait up to the specified time (in seconds) for a file to stop changing before skipping synchronization of the file. Using this option prevents a one-time sync from waiting on a constantly changing file. The file is skipped and reported as an error. Default: 0 (disabled).
--create-dir
Create the source directory, target directory, or both if they do not exist, rather than reporting an error and quitting. Use with -d and -r.
-d ldir, --local-dir=ldir
Synchronize the specified local directory. Use --create-dir to create the remote directory if it does not already exist.
--dedup[=mode]
Take the specified the action when Sync detects duplicate files on the source, even if they have different pathnames. Requires -k. Available modes are hardlink, or copy. Default: hardlink.
  • hardlink - When two or more source files are duplicates, a hardlink is created between them on the target. This saves storage by preventing multiple copies of the same file from accumulating on the target. The files on the target have the same inode, even if the source files have different inodes. The target must be running a Unix-based operating system.
  • copy - After a file is synced on the target, the synced file is copied to the duplicate. This saves bandwidth by not transferring duplicate files. This mode is useful when the target is Windows.

Without the dedup option, all duplicate files are synced. Duplicates might still be synced, rather than hardlinked or copied, if one of the duplicates has not yet been synced on the target.

--delete-delay
Postpone the actual deletion of files or directories until the end of the synchronization.
-E file, --exclude-from=file
Skip paths specified in the filter file. For more information on setting filters, see Include/Exclude Filtering Rules.
--exclude=pattern
Exclude paths that match pattern. Wildcards, such as * and ?, are supported but rules containing them must be in double quotes. For example, --exclude="*.jpg".For more information, see Include/Exclude Filtering Rules.
--exclude-dirs-older-than=mtime
After the initial Sync scan, do not scan directories during subsequent synchronizations if they or their parents have a recursive modified time older than the specified value. The recursive modified time of a directory is the most recent modification time of it or any of its children (file or directory). This option is used to avoid rescanning directories that are known to be unchanged since the previous synchronization, such as a monthly archive directory structure in which only the most recent subdirectory is being modified.

mtime may be specified in any one of the following ways:

  • As a positive number of seconds since 1970-01-01 00:00:00, for Unix and POSIX-compliant operating systems.
    Note: Some file servers, such as Windows NT, use a different epoch for the recursive modified time. In this case, MTIME should be specified as a duration relative to present or UTC timestamp.
  • As a UTC timestamp with the format YYYY-MM-DDTHH:MM:SS, such as 2015-01-01T08:00:00.
  • As a duration formatted as DDd HH:MM:SS or WWw DDd HHh MMm SSs. Directories whose "mtime" is older than Now minus MTIME are not scanned. Input requirements: Leading zero fields and spaces may be omitted. The leftmost fields are optional, but fields to the right of the largest unit specified are required. For example, to exclude directories older than 24 hours, you could specify 1d 0:0:0, 24:00:00, or 24h 00m 00s, but not 1d.

This option does not apply to the root directory.

Note: Sync fails if the first run of async and the next run do not use the same --exclude-dirs-older-than option. If the first run specifies --exclude-dirs-older-than, then the next run must use this option, too. If the first run does not include --exclude-dirs-older-than, then the next run fails if this option is specified.
-G size, --write-block-size=size
Use the specified block size for writing. size=integer K, M, or bytes. Default: 64 MB.
-g size, --read-block-size=size
Set block size for reading. size=integer K, M, or bytes. Default: 64 MB.
-H val, --scan-intensity=val
Scan at the set intensity. val can be vlow, low, medium, high, or vhigh. vlow minimizes system activity. vhigh maximizes system activity by continuously scanning files without rest. Default: medium.
--host=host
Synchronize with the remote host that is specified by hostname or IP address. With this option, the characters "@" and ":" are not treated specially in the argument to -r or --remote-dir. The transfer username cannot be specified as part of the remote directory filepath. Instead, it must be set with --user= or in the environment variable $user (on Windows, %USER%). Allowed forms are as follows:
--remote-dir user@host:/rdir # (old method)  
--user user --remote-dir host:/rdir  
--user user --host host --remote-dir /rdir  
--remote-dir host:/rdir  # (uses $user)  
--host host --remote-dir /rdir # (uses $user)

The following means the same as the first three lines above:

 -r /rdir --user=user --host=host

For backward compatibility, -r A:/rdir for any single letter A is still taken as a Windows path, not as --host A -r /rdir. To specify a one-letter host name A, use an explicit --host=A.

-I file, --include-from=file
Scan and include paths specified in the filter file. For more information, see Include/Exclude Filtering Rules.
-i file, --private-key-path=file
Authenticate with the specified SSH private key file. For information on creating a key pair, see Creating SSH Keys.
--ignore-delete
Do not copy removals to the peer. This option is used mostly with uni-directional syncs. In bi-directional sync, a deletion on one side is ignored but the next time async is run, the file is recopied from the other end. In continuous mode, the file is not recopied until either async is restarted or the file is changed (touched).
--include=pattern
Include paths that match pattern. Wildcards, such as * and ?, are supported but rules containing them must be in double quotes. For example, --include="*.jpg". For more information on how to set include and exclude patterns, see Include/Exclude Filtering Rules.
-j, --preserve-gid
Preserve file owner's gid when synchronizing files between Unix-like operating systems. Requires that async is running as Administrator. Default: disabled.
-K direction, --direction=direction
Transfer in the specified direction. direction can be push, pull, or bidi (bi-directional). Default: push.
-k type, --checksum=type
Calculate the specified checksum type. type can be sha1, md5, sha1-sparse, md5-sparse, or none. A value of none is equivalent to a size check only and async will not detect a change in timestamp. Default: sha1-sparse for local storage, none for object storage.
--keep-dir-local=dir
Move deleted files into dir. The directory must exist (it is not created by --create-dir), and must be outside the synchronization directory (or excluded from the sync using --exclude or --exclude-from), but on the same file system.
--keep-dir-remote=dir
Move the server's deleted files into dir. The directory must exist (it is not created by --create-dir), and must be outside the synchronization directory (or excluded from the sync using --exclude or --exclude-from), but on the same file system.
-l rate, --target-rate=rate
Transfer no faster than the specifed maximum transfer rate. rate=integer G/g, M/m, K/k, or bps. Default: 10 Mbps.
--local-mount-signature=signature file
Verify that the local file system is mounted by the existence of this file. This option increases the time required to synchronize deletes. See also --remote-mount-signature.
-m rate, --min-rate=rate
Attempt to transfer no slower than the specified minimum transfer rate. rate=integer G/g, M/m, K/k, or bps. Default: 200 Kbps.
-N pair, --name=pair
Assign a name for the synchronization session. The value can contain only ASCII alphanumeric, hyphen, and underscore characters. This value is stored in the session cookie and can be used in Aspera Console to identify the transfer session.
Note: -N must precede all session options.
-n action, --symbolic-links=action
Handle symbolic links with the specified method. Permitted values are:

copy - create or update the link at the destination. Not valid for Windows source or destination.

skip - ignore the link altogether.

follow - treat the link as if it were the file or directory it points to, so that at the destination, what was a link is now a copy of the file or directory. Functions as skip if source is Windows.

Default: skip)

--no-preserve-root-attrs
Disable the preservation of attributes on the Sync root.
--no-scan
Skip initial scanning. Use this option with a continuous synchronization if you want to synchronize only new files (files that are added after the start of the async session) but not existing files.
-O port, --udp-port=port
Use the specified UDP port for FASP data transfer. Default: 33001.
-o policy, --overwrite=policy
Overwrite files according to the specified policy, which can be always, older, or conflict. Use with -K push and pull. Default: always for -K push and pull; conflict for -K bidi.
Note: When syncing with object storage, only file size (--checksum=none) can be used to compare files. Thus, using --overwrite=always only overwrites files whose sizes have changed. If the content of a local file is different from a file with the same name in object storage but the files are the same size, the file in object storage is not overwritten. To overwrite files in this case, use --overwrite=older.

--overwrite=older is only accurate if the user also specifies --preserve-time (preserve timestamps).

To resolve conflict and error situations in a uni-directional sync, “touch” the problem files on the source and run async with --overwrite=always. This clears allconflict and error states as the problem files are synced.

-P port, --tcp-port=port
Use the specified TCP port for SSH. port must be a valid numeric IP port. Default: 22.
--pending-max=N
Allow the maximum number of files that are pending transfer to be no more than the specified number. This option acts as a buffer. Default: 2000.
--preserve-access-time
Preserve file access time from the source to the destination. Default: disabled.
--preserve-acls=mode
Preserve Access Control Lists (ACL) data. mode can be native, metafile, or none. Default: none. See also --remote-preserve-acls.
  • native - Preserve attributes using the native capabilities of the file system. This mode is not supported on all file systems. If the destination and source do not support the same native ACL format, async reports and error and exits.
  • metafile- Preserve file attributes in a separate file, named filename.aspera-meta. For example, attributes for readme.txt are preserved in a second file named readme.txt.aspera-meta. These metafiles are platform independent and can be copied between hosts without loss of information. This mode is supported on all file systems.
  • none - Do not preserve attributes. This mode is supported on all file systems.

This feature is only meaningful if both hosts are in a common security domain. If a SID (security ID) in a source file does not exist at a destination, the sync proceeds but no ACL data is saved and the log records that the ACL could not be applied.

Note: Both --preserve-acls and --remote-preserve-acls must be specified in order for the target side of a pull to apply the acls.
--preserve-creation-time
Preserve file creation time from the source to the destination. Valid only on Windows machines. Default: disabled.
--preserve-modification-time
Preserve file modification time from the source to the destination. Default: disabled.
--preserve-time
Same as -t.
--preserve-xattrs=mode
Preserve file extended attributes data (xattr). mode can be native, metafile, or none. Default: none. If the sync is run by a regular user, only user-level attributes are preserved. If run as superuser, all attributes are preserved. See also --remote-preserve-xattrs.
  • native - Preserve attributes using the native capabilities of the file system. This mode is not supported on all file systems. If the destination and source do not support the same native xattr format, async reports and error and exits.
  • metafile- Preserve file attributes in a separate file, named filename.aspera-meta. For example, attributes for readme.txt are preserved in a second file named readme.txt.aspera-meta. These metafiles are platform independent and can be copied between hosts without loss of information. This mode is supported on all file systems.
  • none - Do not preserve attributes. This mode is supported on all file systems.
--proxy proxy_url
Synchronize using the specified proxy address. The proxy URL is specified with the following syntax:

dnat[s]://proxy_username:proxy_password@proxy_ip_address[:port]

where the default port for DNAT is 9091, and for DNATS is 9092. The Proxy password must be specified or the synchronization fails to connect to the Proxy server.

-R rem_log_dir,--remote-logdir=rem_log_dir
Use the specified logging directory on the remote host. The directory is created if it does not already exist. If <async_log_dir> is set in aspera.conf on the server, async initally logs to rem_log_dir but is then redirected to the directory specified for <async_log_dir>.
Note: -R cannot be used if the transfer user is restricted to aspshell.
-r rdir, --remote-dir=rdir
Synchronize the specified directory on the remote host. rdir is [[user@]host:]path. If the target is the remote directory, you can use--create-dir to create the remote directory if it does not already exist.
CAUTION:
If the source and target directories are both on the local host, do not specify a target directory that is inside your source directory.
--remote-mount-signature=signature file
Verify that the remote file system is mounted by the existence of this file. This option increases the time required to synchronize deletes.
--remote-preserve-acls=mode
Like --preserve-acls but used when ACLs are stored in a different format on the remote computer. Defaults to the value of --preserve-acls.
Note: Both --preserve-acls and --remote-preserve-acls must be specified in order for the target side of the pull to apply the acls.
--remote-preserve-xattrs=mode
Like --preserve-xattrs but used attributes are stored in a different format on the remote computer. Defaults to the value of --preserve-xattrs.
--remote-scan-interval=duration
Set the scanning interval of the remote computer. See also --scan-interval.
--remove-after-transfer
Remove source files after they are successfully synchronized.
--scan-dir-rename
Enable the detection of renamed directories and files compared to the previous scan, based on matching inodes. Enable the detection of renamed directories and files compared to the previous scan, based on matching inodes. If a new directory is found on the source and its inode matches that of a previously found directory, it is considered a "rename" and the target directory is renamed accordingly. The source directory is scanned for contents change, and the target directory is updated accordingly. This option can only be used on file systems with persistent inodes, and does not work if inodes are volatile, as is the case with many network-mounted file systems.
--scan-file-rename
Enable the detection of renamed files compared to the previous scan, based on matching inodes. If a new file is found and its inode and last-modified time matches that of a previously found file that does not have multiple hardlinks, it is considered a "rename" and the remote file is renamed accordingly. Can only be used on file systems with persistent inodes, and does not work if inodes are volatile, as is the case with many network- mounted file systems.
Note: If --scan-file-rename is used without --scan-dir-rename, a directory rename causes async to create a new directory and rename its files individually.
--scan-interval=duration
Enable periodic scans during a continuous Sync (a session run with the -C option) on the local host. duration is the interval between periodic scans and can be specified as DDd HH:MM:SS.mmm or WWw DDd HHh MMm SSs XXms XXus. Leading zero fields can be omitted. Spaces can be omitted. A plain number XX is interpreted as SSs (seconds).
--sharing-retry-max=N
Retry synchronizations up to the specifed maximum number after a sharing violation. The interval between retries is the number of seconds specified by --cooloff. Default: 3.
--symbolic-links=action
See -n.
-t, --preserve-time
Preserve file timestamps. Default: disabled.
--transfer-threads=N[:size]
Use the specified number of dedicated transfer threads and optionally specify the file size at which files are assigned groups of threads. The number of threads should not exceed the number of available CPU cores (the lower value of the client and server machines). If no size is specified, infinity is used as an upper bound.

For example, to use two transfer threads to transfer files smaller than or equal to 128 bytes and six transfer threads for all other files (those larger than 128 bytes), use the following options:

--transfer-threads=2:128 --transfer-threads=6
-u, --preserve-uid
Preserve the file owner's uid when synchronizing files between Unix-like operating systems. async must be run as Administrator to use this option. Default: disabled.
--user=user
Authenticate the transfer with the specified username. With this option, the characters "@" and ":" are not treated specially in the argument to -r or --remote-dir.
-W token_string, --token=token_string
Use the specified authorization token. The token type (sync-push, sync-pull, or sync-bidi) must match the direction (push, pull, or bidi) of the requested transfer. The token path must match the remote directory of the requested transfer. If an invalid token is provided, the requested transfer will be denied.
-w pass, --pass=pass
Authenticate the transfer with the specified password.
--write-uid=uid
--write-gid=gid
Write files as the user uid or the group gid. uid and gid can be numeric, or by name. If by name, the name is looked up on the hostperforming the write. Failure to set the uid or gid is logged, but is not an error. The uid or gid is set after ascp completes and before moving the file from the staging directory to the final location.

--write-uid conflicts with --preserve-uid, and --write-gid conflicts with --preserve-gid.

-X size, --rexmsg-size=size
Use the specified size (in bytes) for a retransmission request. Maximum: 1440.
-x, --reset
Clear the async database and rescan the synchronized directories and files to create a fresh database. Default: off.
-Z mtu, --datagram-size=mtu
Use the specified datagram size. Value is an integer. Default: detected-path MTU.