aspera.conf - File System Configuration

The settings in the <file_system> section of aspera.conf include the docroot, file permissions, file handling, filters, and checksum reporting. The absolute path, or docroot, is the area of the file system that is accessible to an Aspera transfer user. The default empty value allows access to the entire file system. You can set one global docroot and then further restrict access to the file system by group or individual user.

Important Configuration Notes:

  • The default server configuration gives users full access to the server's file system with read, write, and browse privileges. Aspera strongly recommends setting a global docroot that is an empty folder and setting global file permissions to false. For a compilation of server security best practices, see Configuring Shares Security.
  • Some Aspera features require a docroot in URI format or require a file restriction instead of a docroot. For more information, see Docroot vs. File Restriction.
Configuration methods: These instructions describe how to manually modify aspera.conf. You can also add and edit these parameters using asconfigurator commands. For more information on using asconfigurator, see User, Group and Default Configurations and run the following command to retrieve a complete default aspera.conf that includes the asconfigurator syntax for each setting:
# /opt/aspera/bin/asuserdata -+
  1. Open aspera.conf from the following location:

    /opt/aspera/etc/aspera.conf

  2. Add or locate the <file_system /> section, as in the following example. 
    <file_system>
   <access>
      <paths>
         <path>
            <absolute peer_ip="ip_address">/path/$(name)</absolute>
                                                       <!-- Absolute Path (conditional) -->
            <absolute>/path/$(name)</absolute>            <!-- Absolute Path -->
            <restrictions>
                <restriction></restriction>       <!-- File Restriction 1 -->
                <restriction></restriction>       <!-- File Restriction 2 -->
            </restrictions>
            <read_allowed>true</read_allowed>          <!-- Read Allowed -->
            <write_allowed>true</write_allowed>        <!-- Write Allowed -->
            <dir_allowed>true</dir_allowed>            <!-- Browse Allowed -->
         </path>
      </paths>
   </access>
   <read_block_size>0</read_block_size>                <!-- Read Block Size -->
   <write_block_size>0</write_block_size>              <!-- Write Block Size -->
   <read_threads>0</read_threads>                       <!–- Number of I/O Read Threads -->
   <write_threads>0</write_threads>                     <!–- Number of I/O Write Threads -->
   <scan_threads>0</scan_threads>                       <!-- Number of Dir Scanning Threads -->
   <meta_threads>0</meta_threads>                       <!-- Number of Metadata Threads -->
   <worker_threads>0</worker_threads>
   <sparse_file>false</sparse_file>                     <!-- Sparse File Checking -->
   <fail_on_attr_error>yes</fail_on_attr_error>         <!-- Behavior on Attr Error -->
   <compression_method>lz4</compression_method>         <!-- Compression Method for File Transfer --> 
   <use_file_cache>true</use_file_cache>               <!-- Use File Cache -->
   <max_file_cache_buffer>0</max_file_cache_buffer>    <!-- Max File Cache Buffer-->
   <resume_suffix>.aspx</resume_suffix>                <!-- Resume Suffix -->
   <symbolic_links>follow,create</symbolic_links>      <!-- Symbolic Link Actions -->
   <preserve_attributes> </preserve_attributes>        <!-- Preserve Attributes -->
   <overwrite>allow</overwrite>                        <!-- Overwrite -->
   <file_manifest>disable</file_manifest>              <!-- File Manifest -->
   <file_manifest_path>path</file_manifest_path>        <!-- File Manifest Path -->
   <file_manifest_inprogress_suffix>.aspera-inprogress</file_manifest_inprogress_suffix>
                                                       <!-- File Manifest Suffix -->
   <pre_calculate_job_size>any</pre_calculate_job_size><!-- Pre-Calculate Job Size -->
   <replace_illegal_chars></replace_illegal_chars>     <!-- Convert Restricted Windows Characters -->
   <storage_rc>
      <adaptive>true</adaptive>                        <!-- Storage Rate Control -->
   </storage_rc>
   <filters>                                           <!-– File Filter Pattern List -->
      <filter>rule1</filter>
      <filter>rule2</filter>
   </filters>
   <file_create_mode> </file_create_mode>              <!-- File Create Mode -->
   <file_create_grant_mask>644</file_create_grant_mask><!-- File Create Grant Mask -->
   <directory_create_mode> </directory_create_mode>    <!-- Directory Create Mode -->
   <directory_create_grant_mask>755</directory_create_grant_mask>
                                                       <!-- Directory Create Grant Mask -->
   <partial_file_suffix>.partial</partial_file_suffix> <!-- Partial File Suffix --> 
   <file_checksum>any</file_checksum>                  <!-– File Checksum Method -->  
</file_system>
  3. Edit settings as needed.
    File System Settings Reference
    Field Description Values Default
    Absolute Path The absolute path, or docroot, is the area of the file system that is accessible to an Aspera transfer user. The default empty value allows access to the entire file system. You can set one global docroot and then further restrict access to the file system by group or individual user. Docroot paths require specific formatting depending on where the transfer server's storage is located.
    Format examples
    • Local storage absolute path:/home/aspera424/movies

      Or using a placeholder for usernames: /home/$(name)

    • Local storage in URI format: file:////home/bear/movies

      URI format is required for server-side encryption-at-rest, but is not supported by the Aspera Watch Service.

    • Cloud or on-premises object storage: see Setting Docroots for Object Storage and HDFS.

    Aspera recommends setting a global docroot to an empty folder or a part of the file system specific to each user. If there is a pattern in the docroot of each user, for example, /sandbox/username, you can use a substitutional string. This allows you to assign an independent docroot to each user without setting it individually for each user. See Setting Up Users for information.

    You can also set multiple docroots and make them conditional based on the IP address from which the connection is made by editing aspera.conf. To do so, edit the absolute path setting by adding the IP address using the following syntax:
    <absolute peer_ip="ip_address">path</absolute>
    		 file path or URI undefined (total access)
    File Restriction
    Note: A configuration (global, group, or user) can have a docroot or a file restriction; configurations with both are not supported.
    A set of file system filters that use "*" as a wildcard and "!" to indicate "exclude". Paths are in URI format; special characters in a URI must be URL-encoded.

    Access to a file is rejected unless the file matches the restrictions, which are processed in the following order:

    • If a restriction starts with "!", the user is not allowed to access any files that match the rest of the restriction.
    • If a restriction does not start with "!", the user can access any file that matches the filter.
    • If one or more restrictions do not start with "!", the user can access any file that matches any one of the no-"!" restrictions.

    Format examples:

    • For a specific folder:

      file:////docs/*

    • For the drive root:

      file:///c*

    • For ICOS-S3 storage:

      s3://my_vault/*

    • To exclude access to key files:

      !*.key
    		 URI undefined (total access)
    Read Allowed Set to true (default) to allow users to transfer files and folders from their docroot.
    • true
    • false
    		 true
    Write Allowed Set to true (default) to allow users to transfer files and folders to their docroot.
    • true
    • false
    		 true
    Browse Allowed Set to true (default) to allow users to browse their docroot.
    • true
    • false
    		 true
    Read Block Size (bytes) Set the maximum number of bytes that can be stored within a block as the block is being transferred from the source disk drive to the receiver. The default of zero causes the Aspera sender to use its default internal buffer size, which may vary by operating system. This is a performance-tuning parameter for an Aspera sender (which only takes effect if the sender is a server). positive integer, where 500MB or 524,288,000 bytes is the maximum block size. 0
    Write Block Size (bytes) Set the maximum bytes within a block that an ascp receiver can write to disk. The default of zero causes the Aspera receiver to use its default internal buffer size, which may vary by operating system. This is a performance-tuning parameter for an Aspera receiver (which only takes effect if the receiver is a server). positive integer, where 500MB or 524,288,000 bytes is the maximum block size. 0
    Number of I/O read threads Set the number of threads the Aspera sender uses to read file contents from the source disk drive. It takes effect on both client and server, when acting as a sender. The default of zero causes the Aspera sender to use its internal default, which may vary by operating system. This is a performance-tuning parameter for an Aspera sender. positive integer 0
    Number of I/O Write Threads Set the number of threads the Aspera receiver uses to write the file contents to the destination disk drive. It takes effect on both client and server, when acting as a receiver. The default of zero causes the Aspera receiver to use its internal default, which may vary by operating system. This is a performance-tuning parameter for an Aspera receiver. positive integer 0
    Number of Dir Scanning Threads Set the number of threads the Aspera sender uses to scan directory contents. It takes effect on both client and server, when acting as a sender. The default of zero causes the Aspera sender to use its internal default. This is a performance-tuning parameter for an Aspera sender. positive integer 0
    Number of Metadata Threads Set the number of threads the Aspera receiver uses to create directories or 0 byte files. It takes effect on both client and server, when acting as a receiver. The default of zero causes the Aspera receiver to use its internal default, which may vary by operating system. This is a performance-tuning parameter for an Aspera receiver. positive integer 0
    Number of Worker Threads Set the number of threads the Aspera sender and receiver use to delete files. This is a performance-tuning parameter. positive integer 0
    Sparse File Checking Set to true to enable sparse file checking, which tells the Aspera receiver to avoid writing zero blocks and save disk space. The default of false to tell the Aspera receiver to write all the blocks. This is a performance-tuning parameter for an Aspera receiver. true or false false
    Behavior on Attr Error

    Set behavior for when operations attempt to set or change file attributes (such as POSIX ownership, ACLs, or modification time) and fail. Setting to yes returns an error and causes the operation to fail. Setting to no logs the error and the operation continues

    		 no or yes yes
    Compression Method for File Transfer Set the compression method to apply to transfers. It applies to both the client and server. lz4, qlz, zlib, or none lz4
    Use File Cache Set to true (default) to enable per-file memory caching at the data receiver. File level memory caching improves data write speed on Windows platforms in particular, but uses more memory. This is a performance tuning parameter for an Aspera receiver.

    Aspera suggests using a file cache on systems that are transferring data at speeds close to the performance of their storage device, and disable it for system with very high concurrency (because memory utilization will grow with the number of concurrent transfers).

    		 true or false true
    Max File Cache Buffer (bytes) Set the maximum size allocated for per-file memory cache (see Use File Cache) in bytes. The default of zero will cause the Aspera receiver to use its internal buffer size, which may be different for different operating systems. This is a performance tuning parameter for an Aspera receiver. positive integer 0
    Resume Suffix Set the file name extension for temporary metadata files used for resuming incomplete transfers. Each data file in progress will have a corresponding metadata file with the same name plus the resume suffix specified by the receiver. Metadata files in the source of a directory transfer are skipped if they end with the sender's resume suffix. text string .aspx
    Symbolic Link Actions Set how the server handles symbolic links. For more information about the actions and the interaction between the server configuration and the client request, see Symbolic Link Handling. Combinations of values are logically ORed before use. For example, use none alone to mean skip, and shut out other options; when both follow and follow_wide are set, the latter is recognized.
    To set a combination of actions globally or for individual users, you must edit the configuration file aspera.conf using the appropriate command:
    # asconfigurator -x "set_node_data;symbolic_links,value"
# asconfigurator -x "set_user_data;user_name,username;symbolic_links,value"
    		 none, create, follow, follow_wide, or any combination of the above delimited by commas follow,create
    Preserve Attributes Set the file creation policy. Set to none to not preserve the timestamps of source files. Set to times to preserve the timestamp of the source files at destination.

    Note: For Limelight storage, only the preservation of modification time is supported.

    		none or times blank (use the client setting)
    Overwrite Set to allow to allow Aspera clients to overwrite existing files on the server, as long as file permissions allow that action.

    If set to deny, clients who upload files to the server cannot overwrite existing files, regardless of file permissions.

    		 allow or deny allow
    File Manifest Set to text to generate a text file "receipt" of all files within each transfer session. Set to disable to not create a File Manifest. The file manifest is a file containing a list of everything that was transferred in a given transfer session. The filename of the File Manifest itself is automatically generated based on the transfer session's unique ID. The location where each manifest is written is specified by the File Manifest Path value. If no File Manifest Path is specified, the file will be generated under the destination path at the receiver, and under the first source path at the sender. text, disable, or none none
    File Manifest Path Specify the location to store manifest files. Can be an absolute path or a path relative to the transfer user's home.

    Note: File manifests can only be stored locally. Thus, if you are using S3, or other non-local storage, you must specify a local manifest path.

    		 text string blank
    File Manifest Suffix Specify the suffix of the manifest file during file transfer. text string .aspera-inprogress
    Pre-Calculate Job Size Set to yes to enable calculating job size before transferring. Set to no to disable calculating job size before transferring. Set to any to follow client configurations. yes, no, or any any
    Convert Restricted Windows Characters To enable the replacement of reserved Windows characters in file and directory names with a non-reserved character, set to the single byte, non-restricted character that will be used for the replacement. Only applies to files written to the local Windows file system; to enable on the peer it must be set on the peer's system. single-byte, non-restricted character blank
    File Create Mode Set the file creation mode (permissions). If specified, create files with these permissions (for example, 0755), irrespective of File Create Grant Mask and permissions of the file on the source computer. Only takes effect when the server is a non-Windows receiver. positive integer (octal) undefined
    File Create Grant Mask Set the mode for newly created files if File Create Mode is not specified. If specified, file modes will be set to their original modes plus the Grant Mask values. Only takes effect when the server is a non-Windows receiver and when File Create Mode is not specified. positive integer (octal) 644
    Directory Create Mode Set the directory creation mode (permissions). If specified, create directories with these permissions irrespective of Directory Create Grant Mask and permissions of the directory on the source computer. Only takes effect when the server is a non-Windows receiver. positive integer (octal) undefined
    Directory Create Grant Mask Set the mode for newly created directories if Directory Create Mode is not specified. If specified, directory modes will be set to their original modes plus the Grant Mask values. Only takes effect when the server is a non-Windows receiver and when Directory Create Mode is not specified. positive integer (octal) 755
    File Filter Pattern List Exclude or include files and directories with the specified pattern in the transfer. Add multiple entries for more inclusion/exclusion patterns. To specify an inclusion, start the pattern with '+ ' (+ and a whitespace). To specify an exclusion, start the pattern with '- ' (- and a whitespace). Two symbols can be used in the setting of patterns:
    • A "*" (asterisk) represents zero to many characters in a string. For example, *.tmp matches .tmp and abcde.tmp.
    • A "?" (question mark) represents a single character. For example, t?p matches tmp but not temp.

    For details on specifying rules, see Using Filters to Include and Exclude Files.

    This option applies only when the server is acting as a client. Servers cannot exclude files or directories uploaded or downloaded by remote clients.

    		 text entries blank
    Partial File Name Suffix Set the filename extension on the destination computer while the file is being transferred. Once the file has been completely transferred, this filename extension is removed.

    Note: This option only takes effect when it is set on the receiver side.

    		 text string blank
    File Checksum Method Set the type of checksum to calculate for transferred files. The content of transfers can be verified by comparing the checksum value at the destination with the value read at the source. For more information, see Reporting Checksums. any, md5, sha1, sha256, sha384, or sha512 any
  4. Save and validate aspera.conf.
    Run the following command to confirm that the XML is correctly formatted and the parameter settings are valid:
    # /opt/aspera/bin/asuserdata -v