Composing an Async Session

Sync has more than 80 options that can be used when composing an async session, but only a few are required, and Aspera recommends using several others. These instructions describe how to compose a bidirectional async session between a Windows client and a Linux server, and includes the required and recommended options in the correct order. You can use the short form or long form (POSIX) option tags and the complete commands using both tag formats are summarized at the end.

For a complete list and descriptions of available options, see the Aspera Sync Command Reference. For configuration and option usage required to synchronize with AWS S3 storage, see Sync with AWS S3.

  1. Confirm that both endpoints have Sync-enabled licenses and that the remote endpoint is running an Aspera transfer server application.
    Run ascp -A in the command line and look for sync2 in the Enabled settings section. On the remote computer, look for Aspera Enterprise Server (for Enterprise Server or Connect Server installations) or Aspera Point-to-Point in the first line of the output.
  2. Begin by invoking async.
    > async
  3. Enter instance options.
    Instance options are used to configure the local (client) computer for the async session and are mostly optional. Aspera recommends that you include -L log_dir (or --alt-logdir=log_dir) to set client-side logging to a directory that you can access, because you might not have permission to access the log in its default location (see Logging). The logging directory must not be in the directory that is being synchronized.

    For example, if the Windows client's username is Morgan, Morgan can use -L to log to a directory in the home folder:

    async -L "C:\Users\Morgan\Aspera jobs\log"

    In this example, the path must be in quotes because the path includes a folder name that contains a space. For more information on path formatting, see Aspera Sync Command Reference.

  4. Name the session by using the -N option (or --name=pair).

    -N pair is required in async commands. The value for pair is a name that uniquely identifies the Sync session and is visible in Aspera Console. -N pair must follow any instance options and must precede all session arguments. Names can only use standard alphanumeric characters, plus "_" and "-" characters.

    For example, name the session job1:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1

    Once you name the session, you enter the session options. Session options define the transfer parameters including authentication, transfer rate and policy, database storage, and the folders to synchronize.

  5. Provide authentication credentials.
    Sync supports three methods of authenticating to the server: SSH key, password, and Basic token. Aspera recommends using SSH keys, unless your server requires a Basic token.
    • SSH key: To use use SSH key authentication, your SSH public key must be configured on the remote server. For instructions on creating keys and setting them up on the server, see the IBM Aspera Enterprise Server Admin Guide. Specify the path to your private key file by using the -i file (or --private-key-path=file) option.
    • Password: The password is the one associated with the Aspera transfer user account on the server. You can provide the password as an environment variable (ASPERA_SCP_PASS) or when prompted after starting the command.
    • Basic token: Basic tokens are used for synchronizing to Aspera products that require access key authentication, such as Aspera Transfer Service (ATS) or Aspera Transfer Cluster Manager (ATC Manager). For instructions on creating the Basic token, see Sync with Basic Token Authorization. You can provide the token as an environment variable (ASPERA_SCP_TOKEN) or in the command line using the -W token_string (or --token=token_string) option.

    For example, use -i and specify the path to Morgan's SSH private key in their home folder:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa

    In this case, the path to the SSH key can use platform-agnostic path separators ( / ) and be entered without quotes around it because it does not have a space in it.

  6. If the local data are stored on a mount or object storage, specify the locations for the local snapshot database.
    The snapshot database cannot be located on CIFS, NFS, or other shared file systems mounted on Linux. If the local files and directories specified in the previous step are on a mount, you must specify a local location using -b db_dir (or --local-db-dir=db_dir). The database must not be in the directory that is being synchronized.

    For example, use -b to store the local snapshot database in Morgan's "Aspera jobs" folder:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa -b "C:\Users\Morgan\Aspera jobs\db"
  7. Set transfer parameters.
    The same transfer rate and transfer policy options that are used to control ascp transfers can be applied to async sessions. Aspera recommends setting a target rate that is based on your available bandwidth and system capabilities. Set the target (maximum) rate using -l rate (or --target-rate=rate).

    For example, use -l to set the target rate to 500 Mbps:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa -b "C:\Users\Morgan\Aspera jobs\db" -l 500m
  8. Specify the local directory for synchronization.
    Enter the local directory using -d ldir (or --local-dir=ldir).

    For example, use -d to set the local directory to Morgan's data folder:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa -b "C:\Users\Morgan\Aspera jobs\db" -l 500m -d c:/users/morgan/data
  9. Specify the transfer username, remote host, and remote directory for synchronization.
    Unlike previous options for which one short option flag was equivalent to one long option flag, when specifying the username, remote host, and remote directory, the short flag option is the equivalent of one to three long option flags. For example, if the username is morgan, the remote host IP address is 10.0.0.1, and the remote directory is /data, then the following options are equivalent to each other:
    -r morgan@10.0.0.1:/data
    --remote-dir=morgan@10.0.0.1:/data
    --user=morgan --remote-dir=10.0.0.1:/data
    --user=morgan --host=10.0.0.1 --remote-dir=/data

    If the name of your remote directory contains an "@", use the --user option so that the "@" is not treated specially in the argument for --remote-dir.

    For example, use -r to set the username, remote host, and remote directory:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa -b "C:\Users\Morgan\Aspera jobs\db" -l 500m -d c:/users/morgan/data -r morgan@10.0.0.1:/data
  10. Specify the locations for the remote Sync log and database.
    On the server, Sync logs to the default location (see Logging) if no location is specified for <async_log_dir> in the server's configuration file. Aspera recommends using -R (or --remote-logdir) to specify a logging location to which you have access. The location must be within your docroot on the server, unless you are synchronizing with AWS S3 object storage. -R is overridden by the server's configuration file. If you are restricted to aspshell on the server, you cannot use this option.

    Aspera also recommends using -B (or --remote-db-dir) to specify a location for the remote Sync database. As with the log file, the location must be within your docroot, it is overridden by <async_db_dir> in the server's configuration file, and you cannot use this option if you are restricted to aspshell.

    As on the local computer, the Sync log and database must not be in a directory that is being synchronized.

    For example, to set the remote log and snapshot database files to Morgan's home folder:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa -b "C:\Users\Morgan\Aspera jobs\db" -l 500m -d c:/users/morgan/data -r morgan@10.0.0.1:/data -R /morgan/async/log -B /morgan/async/db
  11. Specify the synchronization mode.
    Sync can be run in three modes:
    • push: The contents of ldir are synchronized to rdir, with the ldir content overwriting the rdir content, by default (unless the overwrite options are specified otherwise, such as to only overwrite if rdir is older, or never overwrite).
    • pull: The contents of rdir are synchronized to ldir, with the rdir content overwriting the ldir content, by default.
    • bidi (bi-directional): The contents of ldir and rdir are synchronized, with newer versions of files and directories overwriting older versions in either ldir or rdir, by default.
    To synchronize the remote folder with the local folder use -K push (or --direction=push).

    For example, use -K bidi to do a bidirectional sync:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa -b "C:\Users\Morgan\Aspera jobs\db" -l 500m -d c:/users/morgan/data -r morgan@10.0.0.1:/data -R /morgan/async/log -B /morgan/async/db -K bidi
  12. Preserve file attributes.
    When a file or directory is transferred between computers, the file is written to the destination as the transfer user and the file modification time (and creation time on Windows) are reset. Most users prefer to preserve timestamps from the source to the destination by using the -t option.

    For example, use -t timestamps:

    async -L "C:\Users\Morgan\Aspera jobs\log" -N job1 -i c:/users/morgan/.ssh/id_rsa -b "C:\Users\Morgan\Aspera jobs\db" -l 500m -d c:/users/morgan/data -r morgan@10.0.0.1:/data -R /morgan/async/log -B /morgan/async/db -K bidi -t
    Note: When synchronizing between Unix-like operating systems, you can also preserve the user IDs (uid) and group IDs (gid) from the source to the destination by using the options -u -j (equivalent to --preserve-uid --preserve-gid).

    Extended file attributes and ACLs can also be preserved; see the Aspera Sync Command Reference.

Summary

The instructions created the following Sync session, shown using short option flags and POSIX (long) flags.

Note: Each option is shown on a separate line for clarity, but should be entered in the command line as a single line.

Using short-format option flags:

async 
    -L "C:\Users\Morgan\Aspera jobs\log" 
    -N job1 
    -i c:/users/morgan/.ssh/id_rsa 
    -b "C:\Users\Morgan\Aspera jobs\db" 
    -l 500m 
    -d c:/users/morgan/data 
    -r morgan@10.0.0.1:/data
    -R /morgan/async/log 
    -B /morgan/async/db 
    -K bidi 
    -t

Using long-format option flags:

async 
    --alt-logdir="C:\Users\Morgan\Aspera jobs\log" 
    --name=job1 
    --private-key-path=c:/users/morgan/.ssh/id_rsa 
    --local-db-dir="C:\Users\Morgan\Aspera jobs\db" 
    --target-rate=500m 
    --local-dir=c:/users/morgan/data 
    --user=morgan
    --host=10.0.0.1
    --remote-dir=/data
    --remote-logdir=/morgan/async/log 
    --remote-db-dir=/morgan/async/db  
    --direction=bidi 
    --preserve-time

If the session is between Linux computers, it also includes the following session options:

-u
-j

Or using long-format option flags:

--preserve-uid
--preserve-gid