A4 Command Reference

ascp4 Syntax

ascp4 options [[user@]srcHost:]source_file1[,source_file2,...] [[user@]destHost:]target_path

User

The username of the Aspera transfer user can be specified as part of the filepath or with the --user option. If you do not specify a username for the transfer, the local username is authenticated by default.

Source and target paths

• If there are multiple source arguments, then the target path must be a directory.
• To describe filepaths, use single quotes (' ') and forward slashes (/) on all platforms.
• Avoid the following characters in filenames: / \ " : ' ? > < & * |.

URI paths: URI paths are supported, but only with the following restrictions:

• If the source paths are URIs, they must all be in the same cloud storage account. No docroot (download), local docroot (upload), or source prefix can be specified.
• If a destination path is a URI, no docroot (upload) or local docroot (download) can be specified.
• The special schemes stdio:// and stdio-tar:// are supported only on the client. They cannot be used as an upload destination or download source.
• If required, specify the URI passphrase as part of the URI or set it as an environment variable (ASPERA_SRC_PASS or ASPERA_DST_PASS, depending on the direction of transfer).

UNC paths: If the server is Windows and the path on the server is a UNC path (a path that points to a shared directory or file on Windows operating systems) then it can be specified in an ascp4 command using one of the following conventions:

1. UNC path that uses backslashes ( \ )

   If the client is a Windows computer, the UNC path can be used with no alteration. For example, \\192.168.0.10\temp. If the client is not a Windows computer, every backslash in the UNC path must be replaced with two backslashes. For example, \\\\192.168.0.10\\temp.

2. UNC path that uses forward slashes ( / )

   Replace each backslash in the UNC path with a forward slash. For example, if the UNC path is \\192.168.0.10\temp, change it to //192.168.0.10/temp. This format can be used with any client operating system.

Environment Variables

If needed, you can set the following environment variables for use with an ascp4 session.

ASPERA_SCP_PASS=password

The password that is used for SSH authentication of the transfer user.

ASPERA_SCP_COOKIE=cookie

A cookie string that is passed to monitoring services.

ASPERA_SRC_PASS=password

The password that is used to authenticate to a URI source.

ASPERA_DST_PASS=password

Set the password that is used to authenticate to a URI destination.

Ascp4 Options

-A, --version

Display version and license information.

--chunk-size=bytes

Perform storage read/write operations with the specified buffer size. Also use the buffer size as an internal transmission and compression block. Valid range: 4 Kb - 128 Mb.

--compare={size|size+mtime|md5|md5-sparse|sha1|sha1-sparse}method

When using --overwrite and --resume, compare files with the specified method. If the --overwrite method is diff or diff+older, the default --compare method is size.

--compression={none|zlib|lz4}

Compress file data inline. Default: lz4. If set to zlib, --compression-hint can be used to set the compression level.

--compression-hint=num

Compress file data to the specified level when --compression is set to an option that accepts compression level settings (currently only zlib). A lower value results in less, but faster, data compression (0 = no compression). A higher value results in greater, slower compression. Valid values are -1 to 9, where -1 is "balanced". Default: -1.

--delete-after, --delete-after-transfer

After all files are transferred, delete files that exist at the destination but not at the source. Objects on the destination that have the same name but different type or size as objects on the source are not deleted. Requires write permissions on the destination. Do not use with multiple sources, --keepalive, or HTTP fallback.

Using --delete-after can be slower than --delete-before because the destination data set that is used to compare objects can be larger after the transfer.

--delete-before, --delete-before-transfer

Before transfer, delete files that exist at the destination but not at the source. Requires write permissions on the destination. Objects on the destination that have the same name but different type or size as objects on the source are not deleted. Do not use with multiple sources, --keepalive, or HTTP fallback.

Using --delete-before can be faster than --delete-after because the destination data set that is used to compare objects can be smaller before the transfer occurs.

-E pattern

Exclude files or directories from the transfer based on the specified pattern. Use the -N option (include) to specify exceptions to -E rules. Up to 16 -E and -N rules can be specified. Rules are applied in the order in which they are encountered, from left to right. The following symbols can be used in the pattern:

• * (asterisk) represents zero or more characters in a string, for example *.tmp matches .tmp and abcde.tmp.
• ? (question mark) represents a single character, for example t?p matches tmp but not temp.

For details and examples, see Applying Filters to Include and Exclude Files.

Note: When filtering rules are found in aspera.conf, they are applied before rules given on the command line (-E and -N).

--exclude-newer-than=mtime

--exclude-older-than=mtime

Exclude files (but not directories) from the transfer based on when the file was last changed. Positive mtime values are used to express time, in seconds, since the original system time (usually 1970-01-01 00:00:00). Negative mtime values (prefixed with "-") are used to express the number of seconds prior to the current time.

--faspmgr-io

Run ascp4 in API mode using FASP manager I/O. ascp4 reads FASPMGR4 commands from management and executes them. The FASPMGR4 commands are PUT/WRITE/STOP to open/write/close on a file on the server.

--file-list=filename

Transfer the content that is listed in filepath. Only the files and directories are transferred; path information is not preserved at the destination. To read a file list from standard input, use "-" in place of file. UTF-8 file format is supported. If a directory does not exist at the destination, it is created (-d is automatically applied). Each source must be specified on a separate line, for example:

src
src2
...
srcN

Restrictions:

• Paths in file lists cannot use user@host:filepath syntax. You must use --user with --file-list.
• Only one --file-list option is allowed per ascp session. If multiple file lists are specified, all but the last are ignored.
• Only files from the file list are transferred, and any additional source files specified on the command line are ignored.

-h, --help

Display the usage summary.

--host=host

Transfer to the specified host name or address. Requires --mode. This option can be used instead of specifying the host as part of the filename (as hostname:filepath).

-i private_key_file

Authenticate the transfer using public key authentication with the specified SSH private key file (specified with a full or relative path). The private key file is typically in the directory $HOME/.ssh/. If multiple -i options are specified, only the last one is used.

-k {0|1|2|3}

Enable the resumption of partially transferred files at the specified resume level. Default: 0. This option must be specified for your first transfer or it does not work for subsequent transfers. Resume levels:

• -k 0: Always retransfer the entire file (same as --overwrite=always).
• -k 1: Compare file modification time and size and resume if they match (same as --overwrite=diff --compare=size --resume).
• -k 2: Compare sparse checksum and resume if they match (same as --overwrite=diff --compare=md5-sparse --resume).
• -k 3: Compare full checksum and resume if they match (same as --overwrite=diff --compare=md5 --resume).

-L local_log_dir[:size]

Log to the specified directory on the local host rather than the default directory. Optionally, set the size of the log file (default 10 MB).

-l max_rate

Transfer at rates up to the specified target rate. Default: 10 Mbps. This option accepts suffixes "G/g" for Giga, "M/m" for Mega, "K/k" for Kilo, and "P/p/%" for percentage. Decimals are allowed. If this option is not set by the client, the server target rate is used. If a rate cap is set in the local or server aspera.conf, then the rate does not exceed the cap.

-m min_rate

Attempt to transfer no slower than the specified minimum transfer rate. Default: 0. If this option is not set by the client, then the server's aspera.conf setting is used. If a rate cap is set in the local or server aspera.conf, then the rate does not exceed the cap.

--memory=bytes

Allow the local ascp4 process to use no more than the specified memory. Default: 256 MB. See also --remote-memory.

--meta-threads=num

Use the specified number of directory "creation" threads (receiver only). Default: 2.

--mode={send|recv}

Transfer in the specified direction: send or recv (receive). Requires --host.

-N pattern

Protect ("include") files or directories from exclusion by any -E (exclude) options that follow it. Files and directories are specified using pattern. Each option-plus-pattern is a rule. Up to 16 rules can be specified. Rules are applied in the order (left to right) in which they're encountered. Thus, -N rules protect files only from -E rules that follow them. Create patterns using standard globbing wildcards and special characters such as the following:

• * (asterisk) represents zero or more characters in a string, for example *.tmp matches .tmp and abcde.tmp.
• ? (question mark) represents any single character, for example t?p matches tmp but not temp.

For details on specifying patterns and rules, including examples, see Applying Filters to Include and Exclude Files.

Note: Filtering rules can also be specified in aspera.conf. Rules found in aspera.conf are applied before any -E and -N rules specified on the command line.

--no-open

In test mode, do not actually open or write the contents of destination files.

--no-read

In test mode, do not read the contents of source files.

--no-write

In test mode, do not write the contents of destination files.

-O fasp_port

Use the specified UDP port for FASP transfers. Default: 33001.

--overwrite={always|never|diff|diff+older|older}

Overwrite files at the destination with source files of the same name based on the method. Default: always. Use with --compare and --resume. method can be the following:

• always – Always overwrite the file.
• never – Never overwrite the file. If the destination contains partial files that are older or the same as the source files and --resume is enabled, the partial files resume transfer. Partial files with checksums or sizes that differ from the source files are not overwritten.
• diff – Overwrite the file if it is different from the source, depending on the compare method (default is size). If the destination is object storage, diff has the same effect as always.

  If resume is not enabled, partial files are overwritten if they are different from the source, otherwise they are skipped. If resume is enabled, only partial files with different sizes or checksums from the source are overwritten; otherwise, files resume.

• diff+older – Overwrite the file if it is older and different from the source, depending on the compare method (default is size). If resume is not enabled, partial files are overwritten if they are older and different from the source, otherwise they are skipped. If resume is enabled, only partial files that are different and older than the source are overwritten, otherwise they are resumed.
• older – Overwrite the file if its timestamp is older than the source timestamp.

-P ssh-port

Use the specified TCP port to initiate the FASP session. (Default: 22)

-p

Preserve file timestamps for access and modification time. Equivalent to setting --preserve-modification-time, --preserve-access-time, and --preserve-creation-time. Timestamp support in object storage varies by provider; consult your object storage documentation to determine which settings are supported.

On Windows, modification time may be affected when the system automatically adjusts for Daylight Savings Time (DST). For details, see the Microsoft KB article, http://support.microsoft.com/kb/129574.

On Isilon IQ OneFS systems, access time (atime) is disabled by default. In this case, atime is the same as mtime. To enable the preservation of atime, run the following command:

# sysctl efs.bam.atime_enabled=1

--policy={fixed|high|fair|low}

Transfer according to the specified policy:

• fixed – Attempt to transfer at the specified target rate, regardless of network capacity. Content is transferred at a constant rate and the transfer finishes in a guaranteed time. The fixed policy can consume most of the network's bandwidth and is not recommended for most types of file transfers. This option requires a maximum (target) rate value (-l).
• high – Adjust the transfer rate to fully utilize the available bandwidth up to the maximum rate. When congestion occurs, the transfer rate is twice as fast as transfer with a fair policy. This option requires maximum (target) and minimum transfer rates (-l and -m).
• fair – Adjust the transfer rate to fully utilize the available bandwidth up to the maximum rate. When congestion occurs, bandwidth is shared fairly by transferring at an even rate. This option requires maximum (target) and minimum transfer rates (-l and -m).
• low – Adjust the transfer rate to use the available bandwidth up to the maximum rate. Similar to fair mode, but less aggressive when sharing bandwidth with other network traffic. When congestion occurs, the transfer rate is reduced to the minimum rate until other traffic decreases.

If --policy is not set, ascp4 uses the server-side policy setting (fair by default).

--preserve-access-time

Preserve the file timestamps (currently the same as -p).

--preserve-creation-time

Preserve the file timestamps (currently the same as -p).

--preserve-file-owner-gid

--preserve-file-owner-uid

(Linux, UNIX, and macOS only) Preserve the group information (gid) or owner information (uid) of the transferred files. These options require that the transfer user is authenticated as a superuser.

--preserve-modification-time

Preserve the file timestamps (currently the same as -p).

--preserve-source-access-time

Preserve the file timestamps (currently the same as -p).

-q

Run ascp4 in quiet mode. This option disables the progress display.

-R remote_log_dir

Log to the specified directory on the remote host rather than the default directory. Note: Client users that are restricted to aspshell are not allowed to use this option.

--read-threads=num

Use the specified number of storage "read" threads (sender only). Default: 2. To set "write" threads on the receiver, use --write-threads.

--remote-memory=bytes

Allow the remote ascp4 process to use no more than the specified memory. Default: 256 MB.

--resume

Resume a transfer rather than retransferring the content if partial files are present at the destination and they do not differ from the source file based on the --compare method. If the source and destination files do not match, then the source file is retransferred. See -k for another way to enable resume.

--scan-threads=num

Use the specified number of directory "scan" threads (sender only). Default: 2.

--sparse-file

Enable ascp4 to write sparse files to disk. This option prevents ascp4 from writing zero content to disk for sparse files; ascp4 writes a block to disk if even one bit is set in that block. If no bits are set in the block, ascp4 does not write the block (ascp4 blocks are 64 KB by default).

--src-base=prefix

Strip the specified prefix from each source path. The remaining portion of the source path is kept intact at the destination. Available only in send mode. For usage examples, see Ascp File Manipulation Examples.

Use with URIs: The --src-base option performs a character-to-character match with the source path. For object storage source paths, the prefix must specify the URI in the same manner as the source paths. For example, if a source path includes an embedded passphrase, the prefix must also include the embedded passphrase otherwise it will not match.

--symbolic-links={follow|copy|skip}

Handle symbolic links using the specified method. On Windows, the only option is skip. On other operating systems, this option takes following values:

• follow – Follow symbolic links and transfer the linked files. (Default)
• copy – Copy only the alias file. If a file with the same name exists on the destination, the symbolic link is not copied.
• skip – Skip symbolic links. Do not copy the link or the file it points to.

-T

Disable in-transit encryption for maximum throughput.

-u user_string

Define a user string for pre- and post-processing. This string is passed to the pre- and -post-processing scripts as the environment variable $USERSTR.

--user=username

Authenticate the transfer using the specified username. Use this option instead of specifying the username as part of the destination path (as user@host:file).

Note: If you are authenticating on a Windows machine as a domain user, the transfer server strips the domain from the username. For example, Administrator is authenticated rather than DOMAIN\Administrator. For this reason, you must specify the domain explicitly.

--worker-threads=num

Use the specified number of worker threads for deleting files. On the receiver, each thread deletes one file or directory at a time. On the sender, each thread checks for the presences of one file or directory at a time. Default: 1.

--write-threads=num

Use the specified number of storage "write" threads (receiver only). Default: 2. To set "read" threads on the sender, use --read-threads.

For transfers to object or HDFS storage, write threads cannot exceed the maximum number of jobs that are configured for Trapd. Default: 15. To use more threads, open /opt/aspera/etc/trapd/trap.properties on the server and set aspera.session.upload.max-jobs to a number larger than the number of write threads. For example,

# Number of jobs allowed to run in parallel for uploads.
# Default is 15
aspera.session.upload.max-jobs=50

-X rexmsg_size

Limit the size of retransmission requests to no larger than the specified size, in bytes. Max: 1440.

-Z dgram_size

Use the specified datagram size (MTU) for FASP transfers. Range: 296-65535 bytes. Default: the detected path MTU.

As of v3.3, datagram size can be specified on the server by setting <datagram_size> in aspera.conf. The server setting overrides the client setting, unless the client is using a version of ascp that is older than 3.3, in which case the client setting is used. If the pre-3.3 client does not set -Z, the datagram size is the discovered MTU and the server logs the message "LOG Peer client doesn't support alternative datagram size".