Aspera Node File Watcher Action Plugin



This action plug-in provides the ability to detect files and folders matching a certain pattern on a remote Aspera server, via the Aspera Node API.

Saved Parameters Description

  • Name: The name used to identify a saved AsperaNodeFileWatcher configured instance.
  • Comments: Some comments about this saved AsperaNodeFileWatcher configured instance.
  • Aspera Node: The remote node name from defined Remote Nodes.
  • Aspera node IP address: The remote node IP address or domain name.
  • Use ssl?: Use Node API with HTTPS on port 9092. Otherwise use HTTP on port 9091.
  • Node api port: To change the default Node API port.
  • Node user name: Node API user name.
  • Node user password: Node API password.
  • Watch directory: The directory search path for files or folders.
  • File pattern: The detection file name or pattern.
  • Depth max: Maximum depth for the matching files search operation. If not set, default to infinite.
  • Ignore folders?: Folders will be ignored. Only matching files are returned.
  • Polling frequency: Interval in seconds to search for matching files in the watch directory.
  • Check only once: Check for matching file presence and stop processing.
  • Keep on-going: Scanning for matching files is continuous. Otherwise wait for matching files once and stop processing.
  • Return single match?: Return only one matching file at a time. Otherwise return matching files as an array.
  • Trigger on partial: A matching file will be detected even if not stable.
  • Trigger persistence scope: Sets the scope/availability of the trigger output. The default value is 'by workorder group'. The definitions are: 'by workorder group', 'by action template', 'absolute', 'by step', 'by workflow', 'none'

Details:

  • 'by workorder group' - Trigger files are tracked while the workorder is executing and will only be trapped once for a workorder group. If the workorder is restarted (new workorder group), all past triggers will be detected again if still matching the search criteria.

  • 'by workflow' - Same as 'by workorder group' except no triggering will occur upon a 'restart' action for the same workflow.

  • 'by action template' - Allows the tracking and thus non-triggering in another workflow for previously processed trigger files shared across workflows via the same template.

  • 'absolute' - The trigger list is tracked regardless of where it occurs.

  • 'by step' - Similar to 'by workflow', but allows for multiple independent trigger steps within the workflow.

  • 'none' - No persistence at all. If a past trigger file is still present, it will re-trigger if 'keep on-going' is selected. This option is recommended when 'keep on-going' is not selected.

Inputs description

The list of inputs depends on the configuration of the AsperaNodeFileWatcher action template.

Changes in this list of inputs will override any values set in the action template.

  • Watch_directory: The directory search path for files or folders.
  • File_pattern: The detection file name or pattern.
  • node_user_name: Node API user name
  • node_password: Node API password
  • server_address: Remote node name or remote server IP address or remote server domain name

Outputs description

  • File_list: The matching files as an array
  • File_path: A single matching file as an string

Supported Actions

None

Dependencies

None

Operating Instructions

Overview:

This action detects new files or folders matching a certain pattern, located below a watch directory in a remote server. Both the pattern matching and the directory path must be provided by the user. The directory may be searched a single time for a matching file (do not select keep on-going) or can be treated similarly to a hot folder where it is continually scanned for matching files (select keep on-going). Matching file presence can also be checked (select check only once), going on the failure path if no matching file is found.

Instructions:

Populate the watch directory field with the directory to be scanned for trigger files.

  • By default, all sub-directories below the watch directory will be searched for.
  • Wildcards cannot be used in the watch directory (for example, /root is OK but not /root/*).
  • The search depth can be restricted with the 'Depth max' parameter, relative to the watch directory.
  • For example, to detect all files just under the watch directory, use depth max 1, and to detect all files 2 levels down, use depth max 2.
  • Populate the file pattern field with the file name or file pattern used for the trigger.
  • Globbing pattern matching can be used in file pattern.
  • Entering *.txt will result in all files with the .txt extension being detected.
  • Multiple patterns can be entered separated by ',' such as *.txt,*.mxf.
  • For a search of files with txt extension under all directories starting with foo, use foo*/*.txt to retrieve /root/foo/toto.txt and /root/foo/foo2/titi.txt (/root being the watch directory).
  • For a search under the foo directory including bar as a sub-directory, use foo/*/bar/*.txt to retrieve /root/foo/foo1/bar/toto.txt and /root/foo/foo2/bar/bar2/titi.txt.
  • ** has the same meaning as *
  • Other examples: fo? will match /root/foo or /root/fo1 but not /root/foo1, fo[1-9] will match /root/fo1 or /root/fo2 but not /root/foo1

The search is case sensitive.

The remote nodes must be running Aspera Enterprise Server, Connect Server or Point to Point (Linux or Windows) with a valid license as they all use the asperanoded service for Node API calls. The node user name is a Node API account (not an operating system account on the Aspera transfer server). The transfer user (which is an operating system account) associated to the Node API account can be found by running this command on the Aspera server:

# /opt/aspera/bin/asnodeadmin -l 
   List of node user(s): 
                user       system/transfer user                    acls 
   ====================    =======================    ==================== 
           node_user                       xfer       []

The path to configure in the watch directory is relative to the docroot of the transfer user (in this example, xfer).

The transfer user docroot can be found in /opt/aspera/etc/aspera.conf on the remote server such as:

                       <user> 
                        <name>xfer</name> 
                        <file_system> 
                            <access> 
                                <paths> 
                                    <path> 
                                        <absolute>/home/xfer/data</absolute> 
                                    </path> 
                                </paths> 
                            </access> 
                        </file_system> 
                        </user>

For example, a watch directory like /root refers to /home/xfer/data/root on the remote server.

By default, only HTTPS (port 9092) is enabled on the Aspera servers for the Node API. To enable HTTP (port 9091) as well, make sure that /opt/aspera/etc/aspera.conf on the remote server holds this section:

<server>
    <server_name>testchris2.sl.dev.asperacloud.net</server_name>
    <http_port>9091</http_port> 
    <https_port>9092</https_port> 
    <enable_http>true</enable_http> 
    <enable_https>true</enable_https> 
  </server>

Restart asperanoded on the remote server (as root) to enable any change in this aspera.conf port configuration:

service asperanoded restart

Select the checkbox "Keep trigger on-going" if it is necessary to periodically scan for files and folders arriving into the Watch directory and act upon their arrival. "Check only once" and "Keep trigger on-going" are incompatible options (do not select both of them).

If it is OK to trigger on a partial file, select "Triggers on partial file". This will allow triggering the file before it is stable.

When changing "Return single match?", force to save and re-publish the workflow by slightly moving the plugin otherwise the change in outputs (from an array type to a string type and vice versa) may not be registered.