Aspera Client User Guide

RedHat, Debian, Solaris, FreeBSD
Version 2.5.2
Chapter 1 Introduction
Chapter 2 Setting Up
2.1 Upgrade from a Previous Version
2.2 Installation
2.3 Enter the License Key
2.4 Configure the Firewall
2.5 Test Transfer with the Aspera Demo Server
2.6 Optimize the Transfer Performance
2.7 Set Up the Public Key Authentication
Chapter 3 ascp Command Reference
3.1 ascp Usage
3.2 ascp Examples
3.3 File Synchronization with Aspera Sync
Appendix 1 Transfer Policies and Transfer Rate
2 Locating the Log Files
3 Uninstall Aspera Client

1. Introduction

Aspera Client is a file transfer client application supercharged with Aspera's fasp™ file transport technology. It includes the following components:

Features
fasp™ transport server
A transport server that accepts incoming connections.
ascp Command
A command-line transfer program.
Aspera Sync
A command-line synchronization program.

The most up-to-date documentation can be found at http://asperasoft.com/support/documentation/scp-client. For further assistance, please contact us at http://asperasoft.com/support.

2. Setting Up

2.1 Upgrade from a Previous Version

If you have one of the following products installed, follow these steps to prepare the system for upgrade:

Step 1 Backup the old configuration files
Backup the configuration files. Depending on the version of your current install, the file location may vary. To verify the version, use the command:
$ ascp -A
You can find the version number preceding the Aspera SCP version. Backup the specified folders according to the version of your current install:
2.0.2+
/opt/aspera/etc/
User settings, license info
Prior to 2.0.1
/var/opt/aspera/etc/
User settings, license info
Step 2 Close all fasp transfer-related applications and connections.
Before upgrading the application, close the following services: When ready, proceed to the next section to complete the upgrade installation.

2.2 Installation

Follow these steps to set up the application:

Step 1 Download the installer
Download the installer from the link below. Use the credentials provided to your organization by Aspera to access. If needed, contact Aspera Technical Support in determining your firm's access credentials. http://www.asperasoft.com/en/downloads/2
Step 2 Run the installer
With proper administrative permissions, execute the following commands to install the application. Replace the file name accordingly:
RedHat
$ rpm -Uvh aspera-scp-client-[version].rpm
Debian
$ dpkg -i aspera-scp-client-[version].deb
Solaris
$ pkgadd -d aspera-scp-client-[version].pkg
FreeBSD
1. $ cd /usr/ports/misc/compat4x
2. $ make
3. $ make install
4. $ make clean
5. $ cd /(the path to the installer package)
6. $ tar -xzvf aspera-scp-client-[version].tar
7. $ sh aspera-scp-client-[version].sh
8. $ kldload aio
On FreeBSD, reflect the folder name compat4x to your system's settings. An additional input/output processing called AIO (Asynchronous I/O) should be loaded for Aspera fasp™ file transfer.

2.3 Enter the License Key

To install your product license, create the following file and copy the license key string into it:

/opt/aspera/etc/aspera-license

When finished, use this command to verify the license info:

$ ascp -A

2.4 Configure the Firewall

The Aspera transfer products require access through the ports listed in the table below. If you cannot establish the connection, review your local corporate firewall settings and remove the port restrictions:

2.5 Test with Aspera Demo Server

To make sure the software is working properly, follow these steps to test download and upload transfers between your system and the Aspera Demo Server.

Step 1 Download from the Aspera Demo Server
In this example, the transfer uses the following setup:
Server address:
demo.asperasoft.com
Server login:
aspera / demoaspera
File to download:
/aspera-test-dir-large/100MB
Download location:
/tmp/
Transfer Settings:
Fair policy, target rate 10M, minimum rate 1M, encryption disabled
Use the following command to download, press y to accept the server's key, and enter the password demoaspera when prompted:
$ ascp -QT -l 10M -m 1M \
  aspera@demo.asperasoft.com:aspera-test-dir-large/100MB /tmp/
You should see the session messages shown below. Description from left to right:
100MB
The file name
23%
The percentage completed
23MB
The amount transferred
509Kb/s
Current transfer rate
11:59 ETA
Estimated time remaining
Step 2 Upload to the Aspera Demo Server
When the file is downloaded, try uploading the same file back to the Aspera Demo Server. Use the command to upload the file (100MB) to the Aspera Demo Server's /Upload directory. Enter the password demoaspera when prompted:
$ ascp -QT -l 10M -m 1M /tmp/100MB aspera@demo.asperasoft.com:Upload/
Troubleshooting
ascp: command not found
See 2.2 Installation and reinstall.
Invalid license key
See 2.3 Enter the License Key and verify the key.
SSH ... failure exit code 1
See 2.4 Configure the Firewall and review the fasp firewall settings.

2.6 Optimize the Transfer Performance

Aspera's fasp™ transport has no theoretical throughput limit. Other than the network capacity, the transfer speed may be limited by the rate settings and the resources of the computers. This section shows you the hardware upgrade guideline to improve the transfer speed.

Hardware Upgrade Guide
Hard disk
The I/O throughput, the disk bus architecture (e.g. RAID, IDE, SCSI, ATA, and Fiber Channel).
Network I/O
The interface card, the internal bus of the computer.
CPU
Overall CPU performance affects the transfer, especially when encryption is enabled.

2.7 Set Up the Public Key Authentication

Public key authentication is an alternative to password authentication, allowing users to authenticate without entering or storing a password. Setting up public key authentication involves generating a public and private key-pair, and giving the public key to servers you want to transfer with.

To create a key-pair in order to log in into other Aspera servers with the public key authentication, follow these steps:

Step 1 Create a SSH public key-pair
Log in the account that will be initiating transfers, issue following commands. The program will prompt you the public key file name, hit enter to use the default name id_rsa. For a passphrase, you can either enter a password or press return twice to leave it blank:
$ mkdir  ~/.ssh
$ ssh-keygen  -t rsa
Step 2 Provide the public key
Provide your public key file to the administrator of the server you are connecting to. The public key can be found in this path:
(home directory)/.ssh/id_rsa.pub
Step 3 Connect with public key authentication
In this example, the setup is as follow:
User account:
jane
Private key:
~/.ssh/id_rsa
File to send:
my/files
Server address:
10.0.0.2
Upload location:
/space
Transfer settings
Fixed policy, target rate 10000(kbps), encryption disabled
Use the command to establish this connection with the public key and send the file:
$ ascp -T -l 10000  -i ~/.ssh/id_rsa  my/files  jane@10.0.0.2:space

3. ascp Command Reference

ascp is a core command-line program for fasp™ transfers. This section covers the usage and examples.

3.1 ascp Usage

The basic ascp syntax guideline:

The symbols used in the paths:
Use single-quote (' ') and forward-slashes (/) on all platforms
Characters to avoid in the file name:
/   \   "   :   '   ?   >   <   &   *   |

If needed, you can use the command to set the password, token, and cookie in the environment variables:

Password:
export ASPERA_SCP_PASS=the-password
Token:
export ASPERA_SCP_TOKEN=the-token
Cookie:
export ASPERA_SCP_COOKIE=the-cookie
Content Protection Password:
export ASPERA_SCP_FILEPASS=content-protect-password
ascp Synopsis
ascp
[-{ATdpqv}] [-{Q|QQ}] [-l max-rate] [-m min-rate] [-w{f|r} [-K probe-rate]]
[-k {0|1|2|3}] [-i pubkey-file] [-Z dgram-size] [-M mgmt-port]
[-u user-string] [-X rexmsg-size] [-g read-size] [-G write-size]
[-S remote-ascp] [-L local-logdir] [-R remote-logdir][-e pre-post]
[-f config-file] [-C n-id:n-count] [-E pattern1 -E pattern2...]
[-O fasp-port] [-P ssh-port] [-o Option1=x[,Option2=y..]]
[-U {1|2}] [-W token-string] [-y {0|1}] [-j {0|1}]
[-Y key-file] [-I certif-file] [-t port] [-x proxy-server]
[[user@]host1:]source-file [[user@]host2:]target-path
ascp General Options
-A
Display version and license information; then exit.
-T
Disable encryption for maximum throughput.
-d
Create target directory if it doesn't already exist.
-p
Preserve file timestamp.
-q
Quiet flag, to disable progress display.
-v
Verbose mode, print connection and authentication debug messages in the log file.
-{Q|QQ}
Enable fair (-Q) or trickle (-QQ) transfer policy. Use the -l and -m to set the target and minimum rates.
-l max-rate
Set the target transfer rate in Kbps. Default: 10000
-m min-rate
Set the minimum transfer rate in Kbps. Default: 0
-w{r|f}
Test bandwidth from server to client (r) or client to server (f). Currently a beta option.
-K probe-rate
Set probing rate (Kbps) when measuring bottleneck bandwidth.
-k {0|1|2|3}
Enable resuming partially transferred files. Default: 0, always retransfer
-i key-file
Use public key authentication and specify the private key file. Typically, the private key file is in the directory $HOME/.ssh/id_[algorithm].
-Z dgram-size
Specify the datagram size (MTU) for fasp™. By default it uses the detected path MTU.
-M port
Set a management port for monitoring and controlling the transfer.
-u user-string
Apply user string, such as variables for Pre- and Post-Processing, in the transfer.
-X rexmsg-size
Adjust the size in bytes of a retransmission request. Max: 1440.
-g read-size
Set the read block size (in bytes). E.g. 1M for 1 megabyte.
-G write-size
Set the write block size (in bytes), E.g. 1M for 1 megabyte.
-S remote-ascp
Specify the name of the remote ascp binary if different.
-L local-log-dir
Specify a logging directory in the local host, instead of using the default directory.
-R remote-log-dir
Specify a logging directory in the remote host, instead of using the default directory.
-e pre-post
Specify an alternate pre-post command. Use complete path and file name.
-f config-file
Specify an alternate Aspera configuration file other than aspera.conf.
-C n-id:n-count
Use parallel transfer on a multi-node/core system. Specify the node id (nid) and count(ncount) in the format 1:2, 2:2. Assign each participant an independent UDP port.
-E pattern
Exclude files or directories with the specified pattern in the transfer. This option can be used multiple times to exclude many patterns. Up to 16 patterns can be used by using -E.
Two symbols can be used in the pattern: * represents zero to many characters in a string, for example "*.tmp" matches ".tmp" and "abcde.tmp". ? represents one character, for example "t?p" matches "tmp" but not "temp".

-O fasp-port
Set the UDP port used by fasp™ for data transfer. Default: 33001
-P ssh-port
Set the TCP port used for fasp™ session initiation. Default: 22
-o
Advanced ascp options as listed below. Use comma "," to separate:
SkipSpecialFiles=no
Skip special files such as devices and pipes.
yes / no. Default: no
RemoveAfterTransfer=no
Remove source file except folder when finish.
yes / no. Default: no
RemoveEmptyDirectories=no
Remove empty folder on the source.
yes / no. Default: no
PreCalculateJobSize=no
Calculate total size before transfer.
yes / no. Default: no
Overwrite=diff
Overwrite files with the same name.
Default: diff, Takes the following values:
  • always Always overwrite the file.
  • never Never overwrite the file.
  • diff Overwrite if file is different from the source.
  • older Overwrite if file is older than the source.
FileManifest=none
Generate a list of all transferred files information.
none / text. Default: none.
FileManifestPath=(path)
Specify the path to store the manifested file.
text string, Default: undefined
FileCrypt=encrypt
Encrypt or decrypt files. Passphrase is required.
encrypt / decrypt. Default: undefined.
RetryTimeout=(secs)
Specify the timeout duration in seconds, for a retry attempt. Default: undefined
-U {1|2}
Priority when sharing physical or virtual bandwidth cap. 1 for higher priority, 2 for regular. Default:2
-W token-string
Specify the token string for the transfer.
ascp HTTP Fallback Options
-y {0|1}
Enable HTTP Fallback transfer server when UDP connection fails. Set 1 to enable.
0 / 1. Default: 0
-j {0|1}
Encode all HTTP transfers as JPEG files. Set 1 to enable. 0 / 1. Default: 0
-Y key-file
The HTTPS transfer's key file name.
-I certif-file
The HTTPS certificate's file name.
-t port
Specify the port for HTTP Fallback Server.
-x proxy-server
Specify the proxy server address used by HTTP Fallback.

3.2 ascp Examples

1.
Transfer all files in \local-dir\files to 10.0.0.2 with target rate 100 Mbps and encryption OFF:
$ ascp -T -l 100000 /local-dir/files root@10.0.0.2:/remote-dir
2.
Transfer with fair rate policy, with maximum rate 100 Mbps and minimum at 1 Mbps:
$ ascp -TQ -l 100000 -m 1000 /local-dir/files root@10.0.0.2:/remote-dir
3.
To perform a transfer with UDP port 42000:

$ ascp -l 100000 -O 42000 /local-dir/files user@10.0.0.2:/remote-dir
4.
To perform a transfer with the public key authentication, using the key file (home directory)/.ssh/asp1-key:
$ ascp -T -l 10000 -i ~/.ssh/asp1-key local-dir/files root@10.0.0.2:/remote-dir
5.
Enclose the target in double-quotes when spaces are present in the username and remote path:
$ ascp -l 100000 local-dir/files "User Name@10.0.0.2:/remote directory"
6.
Send files to a network shares location \\1.2.3.4\nw-share-dir, through the computer 10.0.0.2:
$ ascp local-dir/files root@10.0.0.2:"//1.2.3.4/nw-share-dir/"
7.
Use parallel transfer on a dual-core system, together transferring at the rate 200Mbps, using UDP ports 33001 and 33002. Two commands are executed in different terminal windows:
$ ascp -C 1:2 -O 33001 -l 100m /file root@10.0.0.2:/remote-dir &
$ ascp -C 2:2 -O 33002 -l 100m /file root@10.0.0.2:/remote-dir
8.
Upload the file space\files to the server 10.0.0.2 with password protection (password: secRet):
$ ASPERA_SCP_FILEPASS=secRet ascp -l 10m -o FileCrypt=encrypt local-dir/files \
  root@10.0.0.2:/remote-dir/
Download from the server 10.0.0.2 and decrypt while transferring:
$ ASPERA_SCP_FILEPASS=secRet ascp -l 10m -o FileCrypt=decrypt \
  root@10.0.0.2:/remote-dir  /local-dir
If the password-protected file is downloaded without descrypting (file1.aspera-env, with aspera-env appended), on the local computer, descrypt the file as file1:
$ ASPERA_SCP_FILEPASS=secRet asunprotect -o file1 file1.aspera-env

3.3 File Synchronization with Aspera Sync

Aspera Sync is a feature that can be used to monitor configured "hot folders" for changes, automatically transferring any new or modified files. It can be used for one-way replication between two locations or simply as a way of forwarding files in your work-flow. Sync runs as a service in the background.

Aspera Sync is a command-line tool that can be used to monitor configured "hot folders" for changes, automatically transferring any new or modified files. It can be used for one-way replication between two locations or simply as a way of forwarding files in your work-flow. Sync runs as a service in the background.

The following shell script is used to execute Aspera Sync:

/opt/aspera/bin/asperasync.sh

You can modify the Aspera Sync transfer settings directly in this file. For example, the Aspera Sync is initiated every 10 seconds with target rate 100Mbps (100000Kbps), you can change them by modifying the values of INTERVAL and TARGETRATE in this file, respectively:

...
INTERVAL=10          # Interval for directory sync is in seconds
TARGETRATE=100000    # The highest rate the sync will try to achieve.
...

3.3.1 asperasync.sh Syntax

To execute the Aspera Sync, use the command asperasync.sh with the following syntax:

asperasync.sh Syntax

$ ASPERA_SCP_PASS=pswd  /opt/aspera/bin/asperasync.sh  src-dir  des-dir  arg

pswd
The password for remote login. Use the environment variable ASPERA_SCP_PASS to set it.
src-dir
The source folder.
des-dir
The destination folder. Either the source or the destination folder has to be a local directory on your computer. The parameter on the remote-side takes the form user@remote-address.
arg
The ascp transfer options for this synchronization. See 3. ascp Command Reference.

3.3.2 Aspera Sync Examples

1.
To start a regular synchronization with the setup:
$ ASPERA_SCP_PASS=1234   /opt/aspera/bin/asperasync.sh  /local-src \
  asp1@10.0.0.5:/remote-dest
2.
To start the same synchronize, with additional ascp command options:
$ ASPERA_SCP_PASS=1234  /opt/aspera/bin/asperasync.sh  /local-src \
  asp1@10.0.0.5:/remote-dest  -l 10000 -k3
3.
To start the same synchronization, and keep the script running when the Terminal session is closed. Start the command with the nohup command, and add a & symbol at the end:
$ ASPERA_SCP_PASS=1234
$ nohup  /opt/aspera/bin/asperasync.sh  /local-src \
  asp1@10.0.0.5:/remote-dest  &
4.
To synchronize from a network shares location, through a remote host, to a local directory:
$ ASPERA_SCP_PASS=1234 /opt/aspera/bin/asperasync.sh \
  asp1@10.0.0.10:"//1.2.3.4/nw-share-src"   /inbox

Appendix 1. Transfer Policies and Transfer Rate

The transfer policy and speed determine how you utilize the network resource for fasp™ file transfers. Four transfer policies described below:

fasp™ Transfer Policies
Fixed
fasp™ attempts to transfer at the specified target rate, regardless of the actual network capacity. This policy transfers at a constant rate and finishes in a guaranteed time. In this mode, a maximum (target) rate value is required.
High
fasp™ monitors the network and adjusts the transfer rate to fully utilize the available bandwidth up to the maximum rate. When congestion occurs, a fasp™ session with high policy transfers at a rate twice of a session with fair policy. In this mode, both the maximum (target) and the minimum transfer rates are required.
Fair
fasp™ monitors the network and adjusts the transfer rate to fully utilize the available bandwidth up to the maximum rate. When other types of traffic builds up and congestion occurs, fasp™ shares bandwidth with other traffic fairly by transferring at an even rate. In this mode, both the maximum (target) and the minimum transfer rates are required.
Low
Similar to Fair mode, the low policy uses the available bandwidth up to the maximum rate, but much less aggressive when sharing bandwidth with other network traffic. When congestion builds up, the transfer rate is decreased all the way down to the minimum rate, until other traffic retreats.

Appendix 2. Locating the Log Files

The log file includes detailed transfer information and can be useful for review and support request. The file can be found in the location:

The Log File Location
RedHat, FreeBSD
/var/log/messages
Debian
/var/log/syslog
Solaris
/var/adm/messages

This example shows you how to separate Aspera's log file from the system's log. Check the syslog.conf manual (man syslog.conf) for precise instructions for your system:

Step 1 Locate the system log settings
Open the file:
/etc/syslog.conf
In the file, find the line that sets the system log. For example:
*.info;mail.none;authpriv.none;cron.none   /var/log/messages
Step 2 Assign the Aspera Log to another file
To separate the Aspera transfer products' log from the system log, do the following modifications:
*.info;mail.none;authpriv.none;cron.none;local0.none;local2.none \
  /var/log/messages
local0.info;local2.info -/var/log/aspera.log
With this modification, the Aspera's log will be stored in the file:
/var/log/aspera.log

Appendix 3: Uninstall Aspera Client

To uninstall the product, use the following commands. For RedHat and Debian, replace the Package-name with the printed name from the first command:

Uninstall the Application
RedHat
$ rpm -qa |grep aspera
$ rpm -e Package-name
Debian
$ dpkg -l "aspera*"
$ dpkg -r Package-name
Solaris
$ pkgrm ASPRclnt
FreeBSD
$ sh /opt/aspera/var/uninstall.sh