Controlling Transfer Order with File Lists

Control the transfer order for files added to a Watch Folder by configuring the Watch Folder to transfer files in packages, which are defined by file lists. The file list determines what files goes into a packaged and the order in which they are transferred.

The last file in a file list is transferred last; it will not be transferred until all other files in the file list have been transferred.

Use file lists with workflows that trigger on the presence of a particular file, but require the presence of other files to function correctly. For example, a workflow that builds a project dependent on the existence of other files fails if the trigger file is transferred before those other files have been transferred. Using a file list ensures the trigger file is not transferred until the rest of the files have been transferred.

Creating a File List

The file list is a simple text file with #aspera-filelist in the first line and a list of files. Console only transfers the last file in the list after the other files have finished transferring. Here is an example file list:

#aspera-filelist
superhero_epic_the_musical-english.mp4
superhero_epic_the_musical-spanish.mp4
superhero_epic_the_musical-chinese.mp4
superhero_epic_the_musical.png
superhero_epic_the_musical-english-trailer.mp4
ADI.XML

Note: The paths listed in a file list are relative to the Watch Folder directory. Paths to other directories are not supported.

Configuration

When creating or editing a Watch Folder in Console, you can configure file list handling in the Packages/Drops section under More Options:

Option	Description	Default	Example
Package timeout	The duration in seconds for which the Watch Folder waits for files defined in the file list. If the required files do not appear within the duration, files with dependencies are not transferred because of unsatisfied dependencies.	`10`	`30`
Final transfer	Defines which file is transferred last. Last file in list: The last file in the package list is transferred last. File list: The file list is transferred last, but the other files are transferred without any specific order.	Last file in list	Last file in list
File list filters	Click the button to add a new filter to identify file lists. You can set a filter to include or exclude files by globbing or by regular expression.		Glob Include`*.package`

Example

One of our workflows monitors a Watch Folder destination directory for new files. When it detects an XML file, it starts a process to move all the files listed in that XML file to another directory and packages those files in a media project according to metadata that the XML also provides. If the XML file is transferred before the other files (which is likely, given its small size), our workflow runs and fails due to missing files. To solve this issue, we include a file list with our media files and the XML file:

#aspera-filelist
superhero_epic_the_musical-english.mp4
superhero_epic_the_musical-spanish.mp4
superhero_epic_the_musical-chinese.mp4
superhero_epic_the_musical.png
superhero_epic_the_musical-english-trailer.mp4
ADI.XML

We name our file list superhero_epic.package.

Next, we configure our Watch Folder to detect the file list by adding a glob filter that looks for files that include the *.package pattern. We set our timeout to 30 seconds for buffer, and keep the default Last file in list, to make sure ADI.XML transfers last.

When we move our files into the monitored source directory on the source node, Watch Folder detects the superhero_epic.package file list and waits 30 seconds before checking that all the files are present in the source directory. Finding all the files present, Watch Folder transfers all the .mp4 files and the .png file to the target directory on the destination node first, and then transfers ADI.XML.

Our workflow detects the the ADI.XML file and successfully performs its task.