Resuming Transfers to Object Storage and HDFS

File transfer resume works differently when the target is object storage, and the process depends on the storage platform. Files are transferred to object storage in parts, which are finalized into a complete object once all parts have uploaded.

Configuring File Resume:

You can configure file resume as for regular transfers using the GUI or command line. In the GUI, go to Connections > File Handling and select Resume incomplete files. Files in object or HDFS storage can be compared using timestamps but not checksums. Thus, in the server GUI, select Compare file attributes from the drop-down menu for When checking files for differences.

To specify the same resume policy in an ascp command, use the option -k 1.

Resume Behavior:

  • Resume does not occur if the file is smaller than one part.
  • In general, a file resumes if at least one part has been transferred to the destination.
  • During a transfer, browsing the storage through the GUI or Node API shows partial transfers as filename.partial files. These may be real files or placeholders, depending on the storage type. Real .partial files are visible with the storage's default browser whereas placeholder .partial files are not.
  • When the upload is complete, real .partial files are deleted and placeholder .partial files are no longer produced.