Local File Watcher Action Plugin

This action plugin creates triggers based on the presence in a specified folder of files matching a specified pattern.

Saved Parameters Description

Name: The name used to identify a saved LocalFileWatcher configured instance.
Comments: Some comments about this saved LocalFileWatcher configured instance.
Keep trigger on-going: Wait for matching file(s), spawn a new workflow instance each time matching file(s) are detected and proceed with the next steps. The new instance will wait for new matching files. If this option is not set, wait for matching files once and proceed with the next steps.
Watch directory: The directory path searched for the existence of a trigger filename or pattern.
Multi folder search?: Check this box if multiple folders are provided in the watch directory field.
Trigger filename or pattern: The file name or pattern utilized as a trigger
Trigger filename base: Portion of trigger file that is stripped from the name, altering the root directory for the trigger file.
Use regex pattern matching: Permits the use of regular expression in the trigger file name or pattern text box.
Filename exclusions: All files whose basename match the provided regex, are filtered out of the result
Full path exclusions: All files whose full name (full path including basename) match the provided regex, are filtered out of the result
Polling frequency: Interval in seconds searching occurs in the watch directory for trigger files.
Triggers on partial file: A trigger file will be detected whether presently being modified or complete.
Stability cool-off time: Optional - The minimum time it takes for a file to be seen unchanged to be considered stable. If not provided, the polling frequency is used.
Ignore 0 bytes files?: Empty files are filtered out of the result if this option is selected
Ignore read-locked files?: In windows file being copied can not be detected as growing, which creates a risk of false positive for the file stability check whenever they are locked. So ignoring locked files avoids those false positive.
Return all matching files as a list: List of all matching trigger are made available as output.
Provide content of trigger file as additional output: Parse the trigger file and make its content available as output.
Remove trigger file: Remove the trigger file from the search directory after detection has been made.
Trigger files archive directory: Full path to a location where trigger file copies will be placed.
Add timestamp to archived copy: Appends '_YYYYMMDD_HHMMSS' to the end of the trigger file name when it is archived.
Trigger persistence scope: Sets the scope/availability of the trigger output. The default value is 'by workorder group'. The definitions are: 'by workorder group', 'by action template', 'absolute', 'by step', 'by workflow', 'none'
Enable aggressive persistence list pruning?: When checked, the persistence list is updated at every scan, not just when a new candidate is found. Checking this option is dangerous as if a network drive becomes unaccessible during a scan, all the files on it will be considered not present any longer (unless a pruning beacon file is provided) and hence will re-trigger when the network drive becomes available again.
Pruning-beacon filepath: Optional - The path to a file whose presence indicates the scan folder is accessible

Details:

'by workorder group' - Trigger files are tracked while the workorder is executing and will only be trapped once for a workorder group. If the workorder is restarted (new workorder group), all past triggers will be detected again if still matching the search criteria.
'by workflow' - Same as 'by workorder group' except no triggering will occur upon a 'restart' action for the same workflow.
'by action template' - Allows the tracking and thus non-triggering in another workflow for previously processed trigger files shared across workflows via the same template.
'absolute' - The trigger list is tracked regardless of where it occurs.
'by step' - Similar to 'by workflow', but allows for multiple independent trigger steps within the workflow.
'none' - No persistence at all. If a past trigger file is still present, it will re-trigger if 'keep on-going' is selected. This option is recommended when 'keep on-going' is not selected.

Inputs Description

The list of inputs depends on the configuration of the LocalFileWatcher action template.

Changes in this list of inputs will override any values set in the action template.

allowMultiple: Scanning for trigger files is continuous and processing will be started whenever they are detected.
archiveDirectory: Full path to a location where trigger file copies will be placed.
archiveWithTimestamp: Appends '_YYYYMMDD_HHMMSS' to the end of the trigger file name when it is archived.
directory: The directory path searched for the existence of a trigger filename or pattern.
fileName: The file name or pattern utilized as a trigger
maxEntries: Has impact only when 'Return all matching files as a list' is selected. At that time, this will limit the size of that listing by the values set in maxEntries.
readFile: Parse the trigger file and make its content available as output.
removeFile: Remove the trigger file from the search directory after detection has been made.
tempo: Interval in seconds searching occurs in the watch directory for trigger files.

Outputs Description

FileContent: The contents of the trigger file
FileName: Based on state of 'Return all matching files as a list' will contain a list of all trigger files or name of oldest matching file

Supported Actions

None

Dependencies

None

Operating Instructions

Overview: This action detects new files matching a certain pattern files matching a certain pattern and located below a directory visible locally. Both the pattern matching and the directory path must be provided by the user. The directory may be searched a single time as part of a work flow or can be treated similarly to a hot folder where it is continually scanned at set time intervals looking for a pre-defined trigger file. Once a trigger file has been detected it may either be parsed to obtain its contents, or have the names in the case of multiple triggers, be collected and output. Trigger files may be archived and/or removed once detected.

Instructions: Populate the Watch directory field with the directory to be scanned for trigger files. If the trigger file can appear in a sub directory directly below this "root" directory, then it is important to place a wild card in the Watch directory name to tell the system that it is necessary to traverse through the sub folder to search for the file. For example, to traverse the first directory the path could be something like "/foo/bar/*". This will instruct the system to search the first directory below "bar" for the trigger file. If the trigger file can be deeper in the tree than the first directory, it is necessary to add a second wild card "**" in order to instruct the system to search further into the sub folder level. Taking the first example, if we change the watch directory to "/foo/bar/**", the system will now search ALL directories below the watch directory for the trigger. If the trigger file can only appear in the root directory, it is not advantageous to use the wildcard for performance reasons.

Examples:

If watch directory = /foo/bar/ and pattern = *.txt the detected files can be /foo/bar/titi.txt but not /foo/bar/bar1/toto.txt and not /foo/bar/bar1/bar2/tutu.txt

If watch directory = /foo/bar/* and pattern = *.txt the detected files can be /foo/bar/bar1/toto.txt but not /foo/bar/titi.txt and not /foo/bar/bar1/bar2/tutu.txt

If watch directory = /foo/bar/** and pattern = *.txt the detected files can be /foo/bar/titi.txt and /foo/bar/bar1/toto.txt and /foo/bar/bar1/bar2/tutu.txt

Users can first check the multi_folder search box, and input the watch directory a string in one of the following three ways:

["/tmp/test", "/opt/aspera/orchestrator/tmp/"]
["<%= f1%>", "<%= f2%>"] and provide values to f1 and f2 in the map inputs screen
<%= f1%>;<%= f2%>and provide values to f1 and f2 in the map inputs screen

Once inputted, the plugin will loop through all folders looking for the file matching the pattern provided

In addition to this, folders that start with a DOT(.) are not searched because by default GLOB ignores the folders starting with DOT. In the example, where the directory is set to /foo/bar/*, and a directory such as /foo/bar/.tmp exists, then the Local File Watcher will not look inside this directory. If the existing directory is /foo/bar/tmp, then that directory will be searched as it is an acceptable replacement for *.

Enter the file name or pattern for the trigger. Regarding file name, don't enter the full path, only the filename. Wild cards are permissible here so entering "*.txt" will result in all files with the ".txt" extension being identified as a trigger.
Select "Use regex pattern matching" when more sophisticated pattern matching using regular expressions is required.
Select the checkbox "Keep trigger on-going" if it is necessary to periodically scan for files arriving into the Watch directory and act upon their arrival.
If it is OK to trigger on a partial file, select "Triggers on partial file". This will allow processing to begin before the trigger file has been completely updated.
Select "Provide Content of Trigger File as additional output" if the contents of the file are required for later processing. Typical example of this may be a JSON or XML file that will require parsing later.
To save copies of the trigger file, put the full path in the "Trigger files archive directory". Note that ownerships and permissions on these files will be set based upon the user executing Orchestrator and may well be owned by root.
Placing a value in the "Trigger file name base" will mask off that path from the trigger file name. The use case is that the trigger files exist in directories below the Watch directory, and an archive directory is specified. If the directory(ies) below the Watch directory are not needed in the archive directory, they can be masked off here.
Select "Add timestamp to archived copy" to append a timestamp at the end of the archived trigger files.
Select "Remove trigger file" if the trigger file should be removed post processing.