Writing Custom Metadata for Objects in Object Storage

Files uploaded to metadata-compatible storage (S3, Google Cloud, Azure, and Swift) can have custom metadata written with them by using the --tags or --tags64 option. The argument is a JSON payload that specifies the metadata, and that is base64 encoded if it is used as an argument for --tags64.

Metadata Behavior

  • All objects that are uploaded in a session have the same metadata.
  • If an upload resumes, the metadata of the original transfer is used.
  • Multi-session transfers must specify the same metadata.
  • Metadata are not retrieved when using ascp to download objects; use the REST API associated with the storage.
  • Transfers to object storages that do not support metadata (such as HDFS and Azure Files) fail if metadata is specified.

Specifying Metadata in JSON

The JSON payload has the general syntax of key-value pairs within a "cloud-metadata" section:

{
    "aspera": {
      "cloud-metadata": [
          {"key1":"value1"},
          {"key2":"value2"},
          ...
      ] } }

Restrictions on key-value pairs:

  • key cannot be ctime, mtime, or atime. These keys are reserved and the transfer fails if they are used.
  • key might be case-sensitive, depending on the destination storage type.
  • The key-value pair must be less than 1024 characters.

Sample Ascp Session with Metadata

> ascp --tags='{"aspera":{"cloud-metadata":[{"location":"skellig"}]}}' --mode=send --user=rey --host=s3.asperasoft.com sourcefile.mov s3://s3.amazonaws.com/project