Aspera Central Watcher Plugin

This plugin provide the ability to sense the initiation or completion of a transfer on an Aspera server.

Saved Parameters Description

Name: The name used to identify a saved AsperaCentralWatcher configured instance.
Comments: Some comments about this saved AsperaCentralWatcher configured instance.
Monitored Aspera node: Provides a selector to pick from defined nodes. If only a node is picked, the default central port (#{AsperaCentralWatcher::DEFAULT_CENTRAL_PORT}) will be used.
Aspera Central Address: Provides the IP address at which the Enterprise Server is located and optionally the port if different from the default (#{AsperaCentralWatcher::DEFAULT_CENTRAL_PORT}). This field is used to have orchestrator generate the wsdl file for the Aspera Central Address specified and populate the URI for that wsdl in the 'Aspera Central Service WSDL Address' field.
Trigger On Empty Files: Check this option if you want to trigger on Empty files(Files with 0 bytes)-Available only for file based transfer
Trigger On Zero Byte Transfers: Check this option if you want any zero byte transfers recorded(If you retransmit a file that is already present, check this option if you want to trigger on them)-Available only for file based transfer
Aspera Central service wsdl address : Provides the location for the WSDL file for the Aspera Central Webservice being queried. As mentioned above, the content for field can be generated.
Keep trigger on-going? : States whether the watcher is ran once only, or everytime the event occurs.
Transfer server address : Enables to filter out the transfers that do not use the specified address as the server. Note: this address must be provided the same way it appears in a transfer. For example if the transfer use localhost, a filter on 127.0.0.1 would not work.
Transfer client address : Same as above but filters on the client address.
Server user list: Filters out all transfers sent to recipient account not in the list provided. Use comma, space or semi-column as separators. i.e "root, admin" or "bob bill doug"
Target filepath pattern: Use regexp matching to only trigger when the pattern is match by one of the file in the session. i.e "^/home/ingest_drop/.*"
Transfer direction: Enables to filter out ourgoing or incoming transfers.
Session Status: Specifies on which session event the watcher is triggered. For example: if 'running'is used the watcher triggers when a transfer is detected as being initiated.
Report transfer sessions with no files?: By default, transfers with no files are ignored, but checking this flag, enables to triggers even when no files are sent.
Session File filter: Applies the given filter to all the files in the session. If atleast one file matches the given filter, it will reject the entire session. (Available only for session based trigger)
Polling frequency: Frequency (in seconds) with which the AsperaCentral Web service is called to check for transfer session statuses. By default, a value of 0 is used (i.e. poll as often as possible)
Trigger persistence scope: Sets the scope/availability of the trigger output. The default value is 'By workorder group'. The definitions are: 'By workorder group', 'By action template', 'absolute', 'By step', 'By workflow', 'None'

Details:

'By workorder group' - Trigger folders are tracked while workorder is executing and will only be trapped once. If the workorder is restarted, all past triggers will be detected again if they are still present.
'By workflow' - Same as 'By workorder group' except no triggering will occur upn a 'restart' action as the workflow is still the same.
'By action template' - Allows the tracking and thus non-triggering in the second workflow for previously processed trigger files shared across workflows that contain the same template,
'absolute' - One trigger list is tracked regardless of the where it occurs,
'By step' - Similar to 'By workflow', but allows for multiple independent trigger steps within the workflow.
'None' – No persistence at all. If a past trigger file is still present, it will re-trigger if ‘keep on-going’ is selected. This option is recommended when ‘keep on-going’ is not selected.

Trigger Types: Sets the event for Trigger Initiation. The default instantiation is on Session.

Details:

'Trigger On Session' - Trigger is initiated for every new session.

'Trigger On Files' - Triggers on all the new files from the previous delta. New trigger is initiated on the new files created after initial trigger.

'Trigger On Single File' - New Trigger is created on every single file.

Inputs Description

In most cases, all the inputs are optional. The exception being when the node to be monitored is not defined (no nodes selected, no address provided). In that case the input central node can be used to specify either the ip address or the name of the remote node. Another possibility is to select __RUNTIME__ as a central node in the template and then set the monitored_node input.

transfer_direction: optional override for the template parameter. Valid values are in "any", "incoming", "outgoing" to be provided as string.
keep_ongoing:

Outputs Description

Depending on the session filter specified, multiple session related information are exposed. NOTE: there might be outputs with empty or nil values depending on the monitored session (ex MD5).

Supported Actions

Cancel

Dependencies

None

Operating Instructions

Note: WSDL found at aperacentral:40001/console/services/soap/FASPSession.wsdl can not be used as is. It assumes webservice calls to be local and thus would not work if orchestrator is running on a different machine than the enterprise server. - use the orchestrator wsdl generator - save a local copy of the wsdl and edit the IP address to reflect the address at which the Enterprise Server is actually found.

Note: This works for Enterprise Server, Connect Server, Faspex Server and Point to Point as they all use Aspera Central to expose their statuses.

Note: Reliable API must be enabled on the Aspera Server being polled, and the IP address polling needs to be enabled as shown below:

<central_server>  
    <address>0.0.0.0</address>
    <port>40001</port>
    <persistent_store>enable</persistent_store>
</central_server>

Note: If you are using async, in the default section of the aspera.conf file, the below tag must be specified for async activity to be reported to Aspera Central.

Example for File Filter and Session Filter:

If ACW needs to be setup to find files only from the folder "/tmp/server1/" then the File Filter can be set as follows.

File Filter Pattern : ^/tmp/server1/.*

Similarly, if even 1 file from the list of the files present in the session contains hdfs//, then we can apply the session filter as (^hdfs//.*) and filter out the entire session.