Configuring Multi-Session Transfers

Enterprise Server and Connect Server can achieve significant performance improvements by using multi-session transfers (also known as parallel transfers and multi-part transfers) on multi-node and multi-core systems.

To enable multi-session transfers, run ascp with the option -C nid:ncount, where nid is the node ID and ncount is the number of nodes or cores. Assign each session (or invocation) its own UDP port.

You can also specify that individual files may be split between multiple sessions by specifying the companion option --multi-session-threshold=threshold. The threshold value specifies, in bytes, the smallest-size file that can be split. That is, files greater than or equal to the threshold will be split, while those smaller than the threshold will not.

A default value for the threshold can be specified in the aspera.conf file by setting <multi-session_threshold_default> in the <default> section. To set a value (in bytes) from the command line, run the following:

> asconfigurator -x "set_node_data;transfer_multi_session_threshold_default,threshold"

The value for multi-session threshold depends on the target rates that a single ascp transfer can achieve on your system for files of a given size and the typical distribution of file sizes for the files in the transfer list. Setting it to 0 (zero) indicates that files should not be split.

The command-line setting overrides the aspera.conf setting. If the client's aspera.conf does not specify a default value for the threshold, then the default is taken from the server's aspera.conf (if specified). If neither --multi-session-threshold nor <multi-session_threshold_default> is specified, then no files are split.

Using Multi-Session to Transfer Between Nodes

The following example shows a multi-session transfer on a dual-core system, which together can transfer at up to 2 Gbps, and each command uses a different UDP port. To run simultaneous ascp transfers, you can run each command from its own terminal window, or run a script in a single terminal, or by back-grounding processes with the shell. No multi-session threshold is specified on the command line or in aspera.conf; therefore, no file splitting occurs.

> ascp -C 1:2 -O 33001 -l 1000m /dir01 10.0.0.2:/remote_dir
> ascp -C 2:2 -O 33002 -l 1000m /dir01 10.0.0.2:/remote_dir

Assuming there are multiple files in dir01, ascp will distribute the files between each command to get the most efficient throughput. If there is only one file in dir01, only one of the commands will actually transfer the file.

In the following example, the multi-session threshold is added to the command line from the example above. This enables file splitting and specifies the threshold size, which is the minimum size of a file that can be split.

> ascp -C 1:2 -O 33001 -l 100m --multi-session-threshold=5242880 /dir01 10.0.0.2:/remote_dir
> ascp -C 2:2 -O 33002 -l 100m --multi-session-threshold=5242880 /dir01 10.0.0.2:/remote_dir

In this case, if there are multiple files in dir01, all files less than 5 MB will be distributed between each command, while all files 5 MB or larger will be split to further achieve a level distribution. If there is only one file in dir01, it will be split if it is 5 MB or larger, otherwise the entire file will be transferred by only one of the commands.

Using Multi-Session to Transfer to an Aspera Transfer Cluster

Note: For cloud transfers, file-splitting is currently only supported for AWS S3.

To transfer to an Aspera Transfer Cluster (ATC), you must ensure that the ATC and the remote client are configured correctly. The transfer request is initiated through the Node API using a curl command to HTTP POST a JSON file with the transfer specifications.

Configuring the Aspera Transfer Cluster

  1. Confirm that <chunk_size> has been set in the ATC to a value equal to or greater than the minimum part size.

    For transfers to cloud storage, files are split into chunks, and the chunk size is specified by <chunk_size> in aspera.conf. The file-splitting must respect a minimum split size for cloud storage, known as a part. If a file is larger than the multi-session threshold but smaller than the chunk/part size, it will not be split.

    To view the cluster's transfer configuration, log into the Aspera Transfer Cluster Manager (ATCM). Select the cluster (in this case jmoore-se-demo-cluster), click the Action button, and select Edit Transfer Configuration.



    Confirm that the text string <transfer><protocol_options><chunk_size>67108864</chunk_size> </protocol_options></transfer> is present.



    If chunk size has not been set, SSH to the instance (for instructions, see the Aspera Transfer Cluster Manager Deployment Guide: Customizing the Cluster Manager or Cluster Node Images) and run the following command:

    # asconfigurator -x "set_node_data;transfer_protocol_options_chunk_size,67108864"

    Confirm that chunk size equals part size by opening the file /opt/aspera/etc/trapd/s3.properties on the ATC and looking for the following lines:

    aspera.transfer.upload.part-size=64MB
    aspera.transfer.gathering-policy=ONE_TO_ONE
  2. Set the scaling policy.

    From the Action drop-down menu, select Edit Auto Scaling Policy. In this example for a 20-node cluster, set Max Nodes and Min Available Nodes to 20, as shown below. Also ensure that Max Start Frequency Count is greater than or equal to the values for Max Nodes and Min Available Nodes.



Configuring the Aspera Client Transfer System

  1. Configure multi-session threshold and URI docroot restriction on the Aspera Enterprise Server or Connect Server.

    To set the multi-session threshold, run the following command. In this example, the value is set to 100000000 bytes:

    # asconfigurator -x "set_node_data;transfer_multi_session_threshold_default,10000000"

    To transfer files to and from cloud storage, you must configure a docroot restriction on your cloud-based transfer server instead of a docroot absolute path. A configuration with both a docroot absolute path (docrooted user) and a restriction is not supported. To set a restriction, run the following command. For more information on setting docroot restrictions for cloud storage, see the Aspera Enterprise Server Admin Guide for Linux: Docroot Restriction for URI Paths. In this example, no restrictions are set on the users by setting the value to *.

    # asconfigurator -x "set_node_data;file_restriction,*"

    These commands create the following lines in aspera.conf.

    <?xml version='1.0' encoding='UTF-8'?>
    <CONF version="2">
        <default>
            <file_system>
                <access>
                    <paths>
                        <path>
                            <restrictions>
                                <restriction>*</restriction>
                            </restrictions>
                        </path>
                    </paths>
                </access>
            </file_system>
            <transfer>
                <multi_session_threshold_default>100000000</multi_session_threshold_default>
            </transfer>
        </default>
        <aaa/>
        ...
    </CONF>
  2. Create a JSON transfer request file.

    In the example file below, named ms-send-80g.json, the file 80g-file is being sent to the ATC jmoore-se-demo-cluster.dev.asperacloud.net where the transfer user is xfer, using token authorization and the multi-session upload cookie.

    {
        "transfer": {
            "remote_host": "jmoore-se-demo-cluster.dev.asperacloud.net",
            "remote_user": "xfer",
            "token": "Basic QVVrZ3VobUNsdjBsdjNoYXAxWnk6dXI0VGZXNW5",
            "target_rate_kbps": 700000,
            "destination_root":"/",
            "multipart": 75,
            "paths": [
                {
                    "source": "/80g-file"
                }
            ],
            "ssh_port": 33001,
            "fasp_port": 33001,
            "direction": "send",
            "overwrite" : "always",
            "cookie": "multi-session upload"
        }
    }

Running and Monitoring the Transfer

  1. Initiate the transfer through the Node API with an HTTP POST of the JSON transfer request using a curl command.

    In the example below, the JSON file ms-send-80g.json is posted by the node user ak_data using the password aspera, from the node server localhost where the HTTPS port is the default 9092.

    $ curl -k -v -X POST -d @ms-send-80g.json https://ak_data:aspera@localhost:9092/transfers 
  2. Monitor transfer progress, bandwidth utilization, and the distribution of transfers for each cluster node.
    On UNIX/Linux systems, view bandwidth utilization from a terminal by running nload on the client system by running the following command. In this example, the vertical scale is set to 10 Gbps and the device is ens1f0.
    $ nload -u g -o 10000000 ens1f0
    The nload report below shows bandwidth utilization at 9+ Gbps:


    In the ATCM UI, selecting Monitor Nodes from the Action drop-down menu shows the transfer distribution and utilization for each of the 20 nodes in the cluster: