Reporting Checksums

Configure IBM Aspera Point-to-Point to report checksums for transferred files.

Internally, Point-to-Point determines the success of transfers by using checksums to verify that file contents at a destination match what was read at the source. Point-to-Point can also be configured to report these checksums to users.

Note: Checksum reporting requires that both the server and client nodes be running Enterprise Server, Connect Server, or Point-to-Point 3.4.2 or higher.

By default, checksum reporting is turned off. The feature can be enabled and configured on the server using any of the following methods:

If used, the command-line option overrides settings in aspera.conf and the GUI.

Each method allows you to enable checksum reporting by selecting or setting the following options:

Additional options in aspera.conf and the GUI allow you to configure where checksum reports should be saved.

Enabling from aspera.conf

Open the aspera.conf file on your server and add the <file_checksum> option to the <file_system> section, as in the example below.

Note: The none option is no longer supported as of 3.4.2. If your aspera.conf file has a <file_checksum> setting of none, transfers will fail with the error "Server aborted Session: Invalid configuration file".

To enable and configure the file manifest where checksum report data will be stored, add settings for <file_manifest> and <file_manifest_path>; for example:

<file_system>
    ...
    <file_checksum>md5</file_checksum>  <!-- Enable checksum reporting (md5, sha1, any) -->
    <file_manifest>text</file_manifest>  <!-- Enable file manifest (text, disable) -->
    <file_manifest_path>/tmp</file_manifest_path>  <!-- Path to manifest file -->
    ...
</file_system>

The following table provides details on the configuration options for checksum reporting:

Conf Option / GUI Config Setting Description Values Default

<file_checksum>

File checksum method

Enable checksum reporting, specifying the type of checksum to calculate for transferred files.

md5, sha1, or any any

<file_manifest>

File Manifest

When set to text a text file "receipt" of all files within each transfer session is generated. If set to disable, no file manifest is created. The file manifest is a file containing a list of everything that was transferred in a given transfer session. The filename of the file manifest itself is automatically generated based on the transfer session's unique ID. text, disable disable

<file_manifest_path>

File Manifest Path

The location where manifest files are to be written. The location can be an absolute path or a path relative to the transfer user's home.

If no path is specified, the file will be generated under the destination path at the receiver, and under the first source path at the sender.

Note: File manifests can only be stored locally. Thus, if you are using S3, or other non-local storage, you must specify a local manifest path.
path name blank

Enabling from the GUI

Click Configuration to open the Server Configuration window. Select the Global, Groups, or Users tab, depending on whether you want to configure checksum reporting for all users, or for a particular group or user. Under the File Handling tab, locate the setting for File checksum method. Check the override box and for the effective value, select md5, sha1, or any.

To enable the file manifest from the GUI, locate the File Manifest setting. Check the override box and set the effective value to text.

Locate the File Manifest Path setting on the line just below. Check the override box and set the effective value to a folder where the manifest files are to be saved.

In the above examples, when files are transferred, the manifest is generated to a text file called aspera-transfer-transfer_id-manifest.txt in the directory /tmp.

For details about the settings for File checksum method, File Manifest, and File Manifest Path, see the table of configuration options in the previous section.

Enabling from the ascp Command Line

To enable checksum reporting on a per-transfer-session basis, run ascp with the --file-checksum=hash option, where hash is sha1, md5, or any.

From the ascp command line, you can also enable the manifest with the option --file-manifest=output where output is either text or none. You can set the path to the manifest file with the option --file-manifest-path=path.

For example:

$ ascp --file-checksum=md5 --file-manifest=text --file-manifest-path=/tmp file aspera_user_1@189.0.202.39:/destination_path

Setting up a Pre/Post-processing Script

An alternative to enabling and configuring the file manifest to collect checksum reporting is to set up a pre/post-processing script to report the values.

The checksum of a successfully transferred file is stored in the pre/post environment variable FILE_CSUM. This environment variable can be used in pre/post scripts to capture file checksums. For example, the following script outputs the checksum to the file /tmp/cksum.log:

#!/bin/bash
if [ $TYPE == File ]; then
    if [ $STARTSTOP == Stop ]; then
        echo "The file is: $FILE" >> /tmp/cksum.log
        echo "The file checksum is: $FILE_CSUM" >> /tmp/cksum.log
        chmod 777 $FILE
    fi 
fi

For information on how to set up pre- and post-processing scripts such as the above and how to use builtin pre/post environment variables, see Pre- and Post-Processing (Prepost).