Aspera Sync Command Reference

Basic syntax:

# async [instance_options] -N pair -d ldir -r [user@host:rdir] [session_options] ...
Note: Transfers started by async can be controlled from the Enterprise Server or Connect Server GUI. Canceling an async transfer from GUI shuts down async.

Required Command Options

Naming the async session: -N pair

-N pair is required in async commands. The value for pair is a name that uniquely identifies the Sync session and is visible in Aspera Console. -N pair must follow any instance options and must precede all session arguments. Names can only use standard alphanumeric characters, plus "_" and "-" characters.

Specifying filepaths and filenames: ldir and rdir

ldir specifies the local directory to be synchronized and rdir specifies the remote directory to be synchronized. File paths and filenames must follow these rules:

Specifying the direction of the sync: -K direction

Sync has three modes of synchronization: push, pull, and bidi.

Using continuous mode: -C

Continuous mode is supported only when the file source is Windows or Linux. See the table below for the operating system requirements for the Sync server and client for the different Sync directions.

Continuous Sync Direction Supported Sync Client OS Supported Sync Server OS
PUSH Linux, Windows All
PULL All Linux, Windows
BIDI Linux, Windows Linux, Windows

One-time synchronization is supported between all operating systems.

The following tables are complete command-line options references. View an abbreviated version from the command line by running:

# async -h

For examples of async commands and output, see Examples of Sync Commands and Output.

Instance Options

Option (- Short form / -- Long form) Description

-A
--version

Display the async version information and license information.

-h
--help

Display help for command-line options.
-D[D..] Debug level. Default is 0.

-q
--quiet

Disable progress display.

-L log_dir
--alt-logdir=log_dir

Specify a logging directory on the local host. If the directory doesn't exist, async creates it for you.

--watchd=datastore:host:port[:domain]

Use asperawatchd connected to the specified Redis for the transfer session. redis is the only accepted datastore for this option.
For example:
--watchd=redis:localhost:31415

The optional domain argument allows you to specify if the domain is other than the default root. For more information see Working with the Aspera Watch Service.

--apply-local-docroot

Prepend the local docroot to the local directory.

Session Options

Option Description

-N pair
--name=pair

Name for the synchronization pair, which can be any string used to identify the connection. This value is stored in the session cookie and can be used in Aspera Console to identify the transfer session's name. It can contain only ASCII alphanumeric, hyphen, and underscore characters.

-N must precede any combination of the options below.

-d ldir
--local-dir=ldir

Set the local sync directory to ldir. You can use --create-dir (described below) to create the remote directory if it does not already exist.

-r rdir
--remote-dir=rdir

Set the remote sync directory to rdir, where rdir=[user@host:path]. You can use the --create-dir option (described below) to create the remote directory if it does not already exist.
CAUTION:
For a local synchronization, do not specify a remote (destination) directory that is located inside your source directory.

--host=host

The name of the remote host. When host is specified with this option, the characters "@" and ":" are no longer treated specially in the argument to -r, so may appear in any such name. If --host=host is specified, the remote username cannot be extracted from the argument for --remote-dir and must be supplied by a --user=name option or in the environment variable $user (on Windows, %USER%). Allowed forms are as follows:
--remote-dir user@host:/rootdir # (old method)  
--user user --remote-dir host:/rootdir  
--host host --user user --remote-dir /rootdir  
--remote-dir host:/rootdir  # (uses $user)  
--host host --remote-dir /rootdir # (uses $user)

The following means the same as the first three lines above:

 -r /rootdir --user=user --host=host

For backward compatibility, -r A:/rootdir for any single letter A is still taken as a Windows path, not as --host A -r /rootdir. To specify a one-letter host name A, use an explicit --host=A.

--user=user The name of the remote user. When --user=user is specified, the character "@" is no longer treated specially in the argument to -r, so may appear in any such name.

-w pass
--pass=pass

Set the passphrase string to the user's password.

-W token_string
--token=token_string

Set the authorization token. The token type (sync-push, sync-pull, or sync-bidi) must match the direction (push, pull, or bidi) of the requested transfer. The token path must match the remote directory of the requested transfer. If an invalid token is provided, the requested transfer will be denied.

-i file
--private-key-path=file

The SSH private key file. For information on creating a key pair, see Creating SSH Keys.

--check-sshfp=fingerprint

Compare fingerprint to remote host key hash, fail on mismatch.

-K direction
--direction=direction

Set the transfer direction. direction can be push, pull, or bidi (bi-directional). (Default: push)

-k type
--checksum=type

Set the checksum type. type can be sha1, md5, sha1-sparse, md5-sparse, or none. A value of none is equivalent to a size check only and async will not detect a change in timestamp. (Default: sha1-sparse for local storage, none for object storage)

-P port
--tcp-port=port

Set the TCP port for SSH. port must be valid numeric IP port. (Default: 22)

-O port
--udp-port=policy

Set the UDP port used by FASP for data transfer. (Default: 33001)

-a policy
--rate-policy=policy

Set the transfer rate policy. policy can be fixed, fair, high, or low. (Default: fair)

-l rate
--target-rate=rate

Set the maximum transfer rate. rate=integer G/g, M/m, K/k, or bps. (Default: 10 Mbps)

-m rate
--min-rate=rate

Set min transfer rate. rate=integer G/g, M/m, K/k, or just bps. (Default: 200 Kbps)

-g size
--read-block-size=size

Set block size for reading. size=integer K, M, or just bytes. (Default: 64MB)

-G size
--write-block-size=size

Set block size using for writing. size=integer K, M, or just bytes. (Default: 64MB)

-H val
--scan-intensity=val

Set the intensity for scanning work. val can be one of vlow, low, medium, high, or vhigh. vlow minimizes system activity. vhigh maximizes system activity by continuously scanning files without rest. (Default: medium)

-R rem_log_dir
--remote-logdir=rem_log_dir

Specify a logging directory on the remote host.

-Z mtu
--datagram-size=mtu

Specify the datagram size (mtu) as an integer. (Default: detected-path MTU)

-X size
--rexmsg-size=size

Set the size (in bytes) of a retransmission request (Maximum: 1440).

-c cipher
--cipher=cipher

Set the encryption algorithm. cipher can be none, AES128, AES192, or AES256. (Default: AES128)

-b ldbdir
--local-db-dir=ldbdir

Local database directory. (Default: .private-asp at the root level of the synchronized directory)

You may relocate the snapshot database to a different location than the default one under the ldir specified with -d. This allows placing the database away from the main data files. This is useful for performance tuning. It is also useful when -d ldir is located on a network share volume that does not reliably support database locking. For further usage information, see Multiple Databases (Snap DB).

-B rdbdir
--remote-db-dir=rdbdir

Remote database directory. Similar to -b above, but for the remote database. For further usage information, see Multiple Databases (Snap DB). (Default: .private-asp at the root level of the synchronized directory)

-I file
--include-from=file

Scan and include paths specified in the filter file. For more information on setting filters, see Include/Exclude Filtering Rules.

-E file
--exclude-from=file

Skip paths specified in the filter file. For more information on setting filters, see Include/Exclude Filtering Rules.

--include=pattern
--exclude=pattern

Include and exclude paths that match pattern. For more information on how to set include and exclude patterns, see Include/Exclude Filtering Rules.
--exclude-dirs-older-than=MTIME

After the initial Sync scan, do not scan directories during subsequent syncs if they or their parents have a recursive modified time older than the specified value. The recursive modified time of a directory is the most recent modification time of it or any of its children (file or directory). This option is used to avoid rescanning directories that are known to have not changed since the previous Sync, such as a monthly archive directory structure in which only the most recent subdirectory is being modified.

MTIME may be specified in any one of the following ways:

  • As a positive number of seconds since 1970-01-01 00:00:00, for Unix and POSIX-compliant operating systems.
    Note: Some file servers, such as Windows NT, use a different epoch for the recursive modified time. In this case, MTIME should be specified as a duration relative to present or UTC timestamp.
  • As a UTC timestamp with the format YYYY-MM-DDTHH:MM:SS, such as 2015-01-01T08:00:00.
  • As a duration formatted as DDd HH:MM:SS or WWw DDd HHh MMm SSs. Directories whose "mtime" is older than Now minus MTIME are not scanned. Input requirements: Leading zero fields and spaces may be omitted. The leftmost fields are optional, but fields to the right of the largest unit specified are required. For example, to exclude directories older than 24 hours, you could specify 1d 0:0:0, 24:00:00, or 24h 00m 00s, but not 1d.

This option does not apply to the root directory.

Note: Sync fails if the first run of async and the next run do not use the same --exclude-dirs-older-than option. If the first run specifies --exclude-dirs-older-than, then the next run must use this option, too. If the first run does not include --exclude-dirs-older-than, then the next run fails if this option is specified.

-C
--continuous

Run continuous synchronization. (Default: disabled)
Note: Continuous mode is supported only when the file source is Windows or Linux. Continuous pulls can be run from any operating system if the source is Windows or Linux. Continuous push can be run only by Windows or Linux. Continuous bidi requires that both the Sync client and server are Windows or Linux.

If you receive an inotify error when attempting to run continuous synchronization, see Error Troubleshooting.

If a file is open, async cannot transfer the file due to sharing violations and may ignore the file if it is closed without changes. To specify the maximum number of retries after a sharing violation, use the --sharing-retry-max=N option. To enable periodic scans that detect when an opened file has been closed and can be transferred, use the --scan-interval=duration option.

--scan-interval=duration Enable periodic scans during a continuous Sync (a session run with the -C option). duration defines the interval between periodic scans. duration may be specified as DDd HH:MM:SS.mmm or WWw DDd HHh MMm SSs XXms XXus. Leading zero fields may be omitted. Spaces may be omitted. A plain number XX will be interpreted as SSs (seconds).
--remote-scan-interval=duration Like --scan-interval but scanning the remote machine.

-t
--preserve-time

Preserve file timestamps. (Default: disabled)
--preserve-access-time Set the access time of the destination file to the same value as that of the source file. (Default: disabled)
--preserve-modification-time Set the modification time of the destination file to the same value as that of the source file. (Default: disabled)
--preserve-creation-time Set the creation time of the destination file to the same value as that of the source file. Only valid on Windows machines. (Default: disabled)

-u
--preserve-uid

-j
--preserve-gid

Preserve file owner's uid or gid. This preserve option requires that async is running as root. (Default: disabled)

--write-uid=uid
--write-gid=gid

Write files as the user uid or the group gid. uid and gid may be numeric, or by name. If by name, the name is looked up on the host actually performing the write. Failure to set the uid or gid is logged, but is not an error. The uid or gid is set after ascp completes and before moving the file from the staging directory to the final location.

--write-uid=uid conflicts with --preserve-uid and --write-gid=gid conflicts with --preserve-gid.

--scan-dir-rename Enable detection of renamed directories and files on the initial scan. Results are only relevant if the nodes for your file system are stable.
--scan-file-rename Enable detection of renamed files on the initial scan. Results are only relevant if the nodes for your file system are stable.

-o policy
--overwrite=policy

Set the overwrite policy to always, older, or conflict. Use with -K push and pull. (Default: always for -K push and pull; conflict for -K bidi)
Note: When syncing with object storage, only file size (--checksum=none) can be used to compare files. Thus, using --overwrite=always only overwrites files whose sizes have changed. If the content of a local file is different from a file with the same name in object storage but the files are the same size, the file in object storage is not overwritten. To overwrite files in this case, use --overwrite=older.

--overwrite=older is only accurate if the user also specifies --preserve-time (preserve timestamps).

To resolve conflict and error situations in a uni-directional sync, “touch” the problem files on the source and run async with --overwrite=always. This clears allconflict and error states as the problem files are synced.

-x
--reset

Clear the async snapshot database and rescan the synchronized directories and files to create a fresh snapshot. (Default: disabled)
--assume-no-mods Assume the directory structure has not been modified. If a directory's modification time has not changed compared to the last snapshot, async in non-continuous mode skips scanning the directory. This option speeds up scanning static directory structures.
--ignore-delete Do not copy removals to the peer. This option is used mostly with uni-directional syncs. In bi-directional sync, a deletion on one side is ignored but the next time async is run, the file is recopied from the other end. In continuous mode, the file is not recopied until either async is restarted or the file is changed (touched).
--delete-delay Postpone the actual deletion of files or directories until the end of the sync session.
--keep-dir-local=dir Move deleted files into dir. Note that dir must exist (it is not created by --create-dir), and must be outside the synchronization directory (or excluded from the sync using --exclude or --exclude-from), but on the same file system.
--keep-dir-remote=dir Move the server's deleted files into dir. Note that dir must exist (it is not created by --create-dir), and must be outside the synchronization directory (or excluded from the sync using --exclude or --exclude-from), but on the same file system.
--local-mount-signature=signature file Verify that the local file system is mounted by the existence of this file. Use of this feature increases the time to synchronize deletes.
--remote-mount-signature=signature file Verify that the remote file system is mounted by the existence of this file. Use of this feature increases the time to synchronize deletes.
--create-dir Create the remote directory if it does not exist. This option is used with the -d and -r options (async creates directories if they do not exist, rather than reporting an error and quitting).
--dedup[=mode] Specify the action to take when Sync detects duplicate files on the source, even if they have different pathnames. This option requires specifying the checksum type with the -k option.

Available modes are hardlink, inode, or copy (Default: hardlink).

  • hardlink - When two or more source files are duplicates, a hardlink is created between them on the target. This saves storage by preventing multiple copies of the same file from accumulating on the target. The files on the target have the same inode, even if the source files have different inodes. The target must be running a Unix-based operating system.
  • inode - When two or more source files have matching inodes, a hardlink is created between them on the target and the target files have matching inodes. This is only supported between Unix-based platforms. If --dedup=inode is used in a continuous sync, Aspera recommends using the scan-interval option.
  • copy - After a file is synced on the target, the synced file is copied to the duplicate. This saves bandwidth by not transferring duplicate files. This mode is useful when the target is Windows.

Without the dedup option, all duplicate files are synced.

Duplicates may still be synced, rather than hardlinked or copied, if one of the duplicates has not yet been synced on the target.

--no-scan Skip initial scanning.
--cooloff=sec Number of seconds to delay the start of the transfer. For example, if --cooloff=5, async waits 5 seconds before copying a file. If --cooloff=0 transfers start immediately. Note that both peers get the same cooloff period. The permitted value for sec is any integer from 0 to 60. (Default: 3)
--cooloff-max=sec Maximum time to wait, in seconds, for a file to stop changing before skipping the file. Using this option prevents a one-time sync from waiting on a constantly changing file. The file is skipped and reported as an error. (Default: 0, in which case the option is disabled)
--pending-max=N This option acts as a buffer to ensure that files to be transferred do not exceed the maximum. (Default: 2000)
--sharing-retry-max=N Maximum number of times to retry after a sharing violation. The interval between retries is the number of seconds specified by --cooloff. (Default: 3) Not supported
--preserve-acls=mode Preserve the ACL access data from Windows or OS X files if the preservation mode is native or metafile. (Default: none)

If mode is native, async expects that the destination supports the same native ACL format as the source; if not, it generates an error and exits.

If mode is metafile, async writes the access data to a separate file in the same location; the file has same name but with the added extension .aspera-meta. Metafile data is in a generic format that can reside on any platform and also be reconverted to native form if the file is again synced with a system that supports it.

If mode is none, no ACL data is preserved at all.

This feature is only meaningful if both hosts are in a common security domain. If a SID (security ID) in a source file does not exist at a destination, the sync proceeds but no ACL data is saved and the log records that the ACL could not be applied.

Note: Both --preserve-acls and --remote-preserve-acls must be specified in order for the target side of the pull to apply the acls.

--preserve-xattrs=mode

Preserve OS X extended attributes data (xattr) if preservation mode is native or metafile. (Default: none)

If the sync is run by a regular user, only user-level attributes are preserved. If run as superuser, all attributes are preserved.

If mode is native, async expects that the destination supports the same native xattr format as the source; if not, it generates an error and exits.

If mode is metafile, async writes the xattr data to a separate file in the same location; the file has the same name with the added extension .aspera-meta. The aspera-meta files are platform-independent and can be copied between hosts without loss of data. They can be reconverted to the native form if the file is again synced with an OS X system.

If mode is none, no xattr data is preserved at all.

--remote-preserve-acls=mode Like --preserve-acls but used when ACLs are stored in a different format on the remote host. Defaults to the value of --preserve-acls.

Note: Both --preserve-acls and --remote-preserve-acls must be specified in order for the target side of the pull to apply the acls.

--remote-preserve-xattrs=mode Like --preserve-xattrs but used attributes are stored in a different format on the remote systgem. Defaults to the value of --preserve-xattrs.
--compression=mode Specify method used to compress a file before transfer. mode can be zlib or none. The default value is none.
--transfer-threads=N[:size] Specify intervals for dedicated transfer threads. N corresponds to the number of threads used to process files smaller or equal to the specified size. The number of threads should not exceed the number of available CPU cores (the lower value of the client and server machines). If no size is specified, infinity is used as an upper bound.

For example, to use 2 transfer threads to transfer files with a size smaller than 128 bytes and all other files with 6 transfer threads, use the following options:

--transfer-threads=2:128 --transfer-threads=6
--remove-after-transfer Remove source files after they are successfully synchronized.
--no-preserve-root-attrs Disable preservation of attributes on Sync root.