Transfers with Aspera On Demand and Object-Storage-Based Aspera Servers

Tranfers with Aspera On Demand servers or Enterprise Servers located in object storage must provide credentials to the object storage in one of the following ways:

  • Specify the storage password or secret key in the transfer user's docroot. (Preferred method)
  • Set the storage password or secret key as an environment variable.
  • Specify the storage password or secret key in the command line.

With Docroot Configured: Authenticate in the Docroot

If your transfer user account has a docroot set that includes credentials or credentials are configured in the .properties file, ascp transfers to and from Alibaba Cloud, Amazon S3, IBM COS - S3, Google Cloud Storage, Akamai, Softlayer, Azure, and are the same as regular ascp transfers. For command syntax examples, see Ascp General Examples.

For instructions on configuring a docroot for these types of storage, see Aspera Enterprise Server Admin Guide (Linux): Docroot Path Formatting for Cloud, Object, and HDFS Storage. You are prompted for the transfer user's password upon running these commands unless you have set the ASPERA_SCP_PASS environment variable or are using an SSH key, as described previously.

With No Docroot Configured: Authenticate with Environment Variables

You can set an environment variable (ASPERA_DEST_PASS) with the storage password or access key using the command below:

$ export ASPERA_DEST_PASS = secret_key

With this and ASPERA_SCP_PASS set, run ascp with the syntax listed in the table above, but you do not need to include the storage password or access key, and are not prompted for the Aspera password upon running the command.

Note: The ASPERA_DEST_PASS variable is not applicable to Google Cloud Storage or Amazon S3 using IAM roles.

With No Docroot Configured: Authenticate in the Command Line

If you do not have a docroot configured and do not set an environment variable (described previously), you must authenticate in the command line. In the examples below, you include the storage password or secret key as part of the destination path. You are prompted for the transfer user's password upon running these commands unless you have set the ASPERA_SCP_PASS environment variable or are using an SSH key, as described above.

Storage Platform ascp Syntax and Examples
Alibaba Cloud Aspera recommends running ascp transfers with Alibaba Cloud with a docroot configured.
Amazon S3
  • If you are using IAM roles, you do not need to specify the access ID or secret key for your S3 storage.
Upload syntax:
$ ascp options --mode=send --user=username --host=s3_server_addr source_files s3://access_id:secret_key@s3.amazonaws.com/my_bucket

Upload example:

$ ascp --mode=send --user=bear --host=s3.asperasoft.com bigfile.txt s3://1K3C18FBWF9902:GEyU...AqXuxtTVHWtc@s3.amazonaws.com/demos2014

Dowload syntax:

$ ascp options --mode=recv --user=username --host=s3_server_addr s3://access_id:secret_key@s3.amazonaws.com/my_bucket/my_source_path destination_path

Download example:

$ ascp --mode=recv --user=bear --host=s3.asperasoft.com s3://1K3C18FBWF9902:GEyU...AqXuxtTVHWtc@s3.amazonaws.com/demos2014/bigfile.txt /tmp/
Azure These examples are for Azure blob storage. For Azure Files, use the syntax: azure-files://storage_account:storage_access_key@file.core.windows.net/share. Aspera recommends running ascp transfers with Azure Data Lake Storage with a docroot configured.

Upload syntax:

$ ascp options --mode=send --user=username --host=server_address source_files azu://storage_account:storage_access_key@blob.core.windows.net/path_to_blob

Upload example:

$ ascp --mode=send --user=AS037d8eda429737d6 --host=dev920350144d2.azure.asperaondemand.com bigfile.txt azu://astransfer:zNfMtU...nBTkhB@blob.core.windows.net/abc

Dowload syntax:

$ ascp options --mode=recv --user=username --host=server azu://storage_account:storage_access_key@blob.core.windows.net/path_to_blob/source_file destination_path

Download example:

$ ascp --mode=recv --user=AS037d8eda429737d6 --host=dev920350144d2.azure.asperaondemand.com azu://astransfer:zNfMtU...nBTkhB@blob.core.windows.net/abc /downloads
Google Cloud Storage Note: The examples below require that the VMI running the Aspera server is a Google Compute instance. 
$ ascp options --mode=send --user=username --host=server_address source_files gs:///my_bucket/my_path

Upload example:

$ ascp --mode=send --user=bear --host=10.0.0.5 bigfile.txt gs:///2017_transfers/data

Dowload syntax:

$ ascp options --mode=recv --user=username --host=server gs:///my_bucket/my_path/source_file destination_path

Download example:

$ ascp --mode=recv --user=bear --host=10.0.0.5 gs:///2017_transfers/data/bigfile.txt /data
HDFS Aspera recommends running ascp transfers with HDFS with a docroot configured.
IBM COS - S3 Upload syntax:
$ ascp options --mode=send --user=username --host=server_address source_files s3://access_id:secret_key@accessor_endpoint/vault_name

Upload example:

$ ascp --mode=send --user=bear --host=s3.asperasoft.com bigfile.txt s3://3ITI3OIUFEH233:KrcEW...AIuwQ@38.123.76.24/demo2017

Dowload syntax:

$ ascp options --mode=send --user=username --host=server_address s3://access_id:secret_key@accessor_endpoint/vault_name/source_files destination_path

Download example:

$ ascp --mode=send --user=bear --host=s3.asperasoft.com s3://3ITI3OIUFEH233:KrcEW...AIuwQ@38.123.76.24/demo2017 /tmp/
IBM Cloud Object Storage (COS) - Swift Aspera recommends running ascp transfers with IBM Cloud Object Storage (COS) - Swift with a docroot configured.
OpenStack Swift Upload syntax:
$ ascp options --mode=send --user=username --host=ip_addr source_files swift://account_id:api_key@auth_url/my_bucket

Example Upload:

$ ascp --mode=send --user=bear --host=192.155.218.130 bigfile.txt swift://XYZO...46-2:bob:437e...bc16@sjc01.objectstorage.service.networklayer.com/test

Dowload syntax:

$ ascp options --mode=recv --user=username --host=ip_addr swift://account_id:api_key@auth_url/my_bucket/my_source_path destination_path

Download example:

$ ascp --mode=recv --user=bear --host=192.155.218.130 swift://XYZO...46-2:bob:437e29...f616@sjc01.objectstorage.service.networklayer.com/test/bigfile.txt /tmp/
Note: Swift requires additional Trapd configuration settings that can be included as queries attached to the docroot, with the format docroot?setting.

For example, for an upload to IBM COS - Swift, the path is written as follows:

swift://XYZO...46-2:bob:437e...bc16@sjc01.objectstorage.service.networklayer.com/test?aspera.swift.endpoint.auth-path=/auth/v1.0