Using Standard I/O as the Source or Destination

ascp can use standard input (stdin) as the source or standard output (stdout) as the destination for a transfer, usually managed by using the Aspera FASP Manager SDK. The syntax depends on the number of files in your transfer: for single files use stdio:// and for multiple files use stdio-tar://. The transfer is authenticated using SSH or a transfer token.

Named Pipes

A named pipe may be specified as a stdio destination, with the syntax stdio:///path for single files, or stdio-tar:///path for multiple files, where path is the path of the named pipe. If a docroot is configured on the destination, then the transfer goes to the named pipe docroot/path.
Note: Do not use stdio:///path to transfer multiple files. The file data is asynchronously concatenated in the output stream and might be unusable. Use stdio-tar:///path instead, which demarcates multiple files with headers.

Single File Transfers

To upload data that is piped into stdin, set the source as stdio:///?fsize, wherefsize is the number of bytes (as a decimal) that are received from stdin. The destination is set as the path and filename. The file modification time is set to the time at which the upload starts. Standard input must transfer the exact amount of data that is set by fsize. If more or less data is received by the server, an error is generated.

To download data and pipe it into stdout, set the destination as stdio://.

Restrictions:

  • stdio:// cannot be used for persistent sessions. Use stdio-tar:// instead.
  • Only --overwrite=always or --overwrite=never are supported with stdio://. The behavior of --overwrite=diff and --overwrite=diff+older is undefined.

Single-file Transfer Examples:

  • Upload 1025 bytes of data from the client stdin to /remote-dir on the server at 10.0.0.2. Save the data as the file newfile. Transfer at 100 Mbps.
    file_source | ascp -l 100m --mode=send --user=username --host=10.0.0.2 stdio:///?1025 /remote-dir/newfile
  • Download the file remote_file from the server at 10.0.0.2 to stdout on the client. Transfer at 100 Mbps.
    ascp -l 100m --mode=recv --user=username --host=10.0.0.2 remote_file stdio://
  • Upload the file local_file to the server at 10.0.0.2 to the named pipe /tmp/outpipe. Transfer at 100 Mbps.
    ascp -l 100m --mode=send --user=username --host=10.0.0.2 local_file stdio:////tmp/outpipe

Multi-File Transfers

ascp can transfer one or more files in an encoded, streamed interface, similar to single file transfers. The primary difference is that the stream includes headers that demarcate data from individual files.

To upload files that are piped into stdin, set the source as stdio-tar://. The file modification time is set to the time at which the upload starts.

The file(s) in the input stream must be encoded in the following format. File can be the file name or file path, Size is the size of the file in bytes, and Offset is an optional parameter that sets where in the destination file to begin overwriting with the raw inline data:
[0 - n blank lines] 
File: /path/to/file_1 
Size: file_size
Offset: bytes 
 
file_1 data 
[0 - n blank lines] 
File: /path/to/file_2 
Size: file_size 
 
file 2 data 
...

To download one or more files to stdout, set the destination as stdio-tar://. Normal status output to stdout is suppressed during downloads because the transfer output is streamed to stdout. The data sent to stdout has the same encoding as described for uploads.

To download to a named pipe, set the destination to stdio-tar:////path, where path is the path of the named pipe.

When an offset is specified, the bytes that are sent replace the existing bytes in the destination file (if it exists). The bytes added to the destination file can extend beyond the current file size. If no offset is set, the bytes overwrite the file if overwrite conditions are met.

Restrictions:

  • When downloading to stdio-tar://, the source list must consist of individual files only. Directories are not allowed.
  • Only --overwrite=always or --overwrite=never are supported with stdio-tar://. The behavior of --overwrite=diff and --overwrite=diff+older is undefined.
  • Offsets are only supported if the destination files are located in the native file system. Offsets are not supported for cloud destinations.

Multi-file Transfer Examples:

  • Upload two files, myfile1 (1025 bytes) and myfile2 (20 bytes), to /remote-dir on the server at 10.0.0.2. Transfer at 100 Mbps.
    type sourcefile | ascp -l 100m --mode=send --user=username --host=10.0.0.2 stdio-tar:// /remote-dir
    Where sourcefile contains the following:
    File: myfile1 
    Size: 1025 
    
    << 1025 bytes of data>> 
    File: myfile2 
    Size: 20
    
    <<20 bytes of data>>
  • Uploading multiple files from stdin by using a persistent session is the same as a non-persistent session.
  • Update bytes 10-19 in file /remote-dir/myfile1 on the server at 10.0.0.2 at 100 Mbps.
    type sourcefile | ascp -l 100m --mode=send --user=username --host=10.0.0.2 stdio-tar:// /remote-dir
    Where sourcefile contains the following:
    File: myfile1 
    Size: 10
    Offset: 10 
    
    << 10 bytes of data>> 
  • Upload two files, myfile1 and myfile2, to the named pipe /tmp/mypipe (streaming output) on the server at 10.0.0.2. Transfer at 100 Mbps.
    ascp -l 100m --mode=send --user=username --host=10.0.0.2 myfile1 myfile2 stdio-tar:////tmp/mypipe
    This sends an encoded stream of myfile1 and myfile2 (with the format of sourcefile in the upload example) to the pipe /tmp/mypipe. If /tmp/mypipe does not exist, it is created.
  • Download the files from the previous example from 10.0.0.2 to stdout. Transfer at 100 Mbps.
    ascp -l 100m --mode=recv --user=username --host=10.0.0.2 myfile1 myfile2 stdio-tar://
    Standard output receives data identical to sourcefile in the upload example.
  • Download /tmp/myfile1 and /tmp/myfile2 to stdout by using a persistent session. Start the persistent session, which listens on management port 12345:
    ascp -l 100m --mode=recv --keepalive -M 12345 --user=username --host=10.0.0.2 stdio-tar://

    Send the following in through management port 12345:

     FASPMGR 2
     Type: START
     Source: /tmp/myfile1
     Destination: mynewfile1
    
     FASPMGR 2
     Type: START
     Source: /tmp/myfile2
     Destination: mynewfile2
    
     FASPMGR 2
     Type: DONE
    The destination must be a filename; file paths are not supported.

    Standard out receives the transferred data with the following syntax:

    File: mynewfile1 
    Size: file_size 
    
    mynewfile1_data 
    File: mynewfile2 
    Size: file_size
    
    mynewfile2_data
  • Upload two files, myfile1 and myfile2, to named pipe /tmp/mypipe on the server at 10.0.0.2. Transfer at 100 Mbps.
    ascp -l 100m --mode=send --user=username --host=10.0.0.2 myfile1 myfile2 stdio-tar:////tmp/mypipe

    If file/tmp/mypipe does not exist, it is created.

  • Upload two files, myfile1 (1025 bytes) and myfile2 (20 bytes) from stdio and regenerate the stream on the destination to send out through the named pipe /tmp/mypipe on the server at 10.0.0.2. Transfer at 100 Mbps.
    type sourcefile | ascp -l 100m --mode=send --user=username --host=10.0.0.2 stdio-tar:// stdio-tar:////tmp/pipe
    Where sourcefile contains the following:
    File: myfile1 
    Size: 1025 
    
    << 1025 bytes of data>> 
    File: myfile2 
    Size: 20
    
    <<20 bytes of data>>