Include/Exclude Filtering Rules

Notes on Separators and Case

Specifying Rules on the Command Line

Rules may be specified on the command line (for example, --exclude PATTERN) or read from a file (for example, --exclude-from=FILE) to make async skip or include specific directories and files. The following command-line options can be intermixed:

Rules are applied in the order that they are encountered, and the first matching rule (whether including or excluding) takes precedence. For example, if the file aspera-test specifies rules W, X and Y, then the following options apply rules V, W, X, Y, Z (in that order).

--include V --exclude-from aspera-test --include Z

Directories and files are visited in strict depth order, so that with the command-line options --exclude /above/ --include /above/below the file /above/below is never considered. The directory /above/ is not scanned; thus, "/above/below" has no opportunity to be checked against any rules.

Note: If the filtering rule includes a wildcard, such as *, the rule must be in double quotes. For example, --exclude "*.jpg".

Specifying Rules in aspera.conf

Rules may be specified in aspera.conf, as they are for ascp sessions, to apply to sessions run by a specific user or all users. These rules are applied first, then any command line filters are applied.

CAUTION:
Filters set in aspera.conf apply to both ascp and async sessions. If you do not want async filtering rules to apply to ascp sessions, set the rules for a specific user and use that user for async sessions. If you notice your async sessions are being filtered in unexpected ways, search aspera.conf for <filters> to determine what rules have been configured. You can find aspera.conf in the following location:

C:\Program Files (x86)\Aspera\Point-to-Point\etc\aspera.conf

To specify an inclusion, start the filter pattern with '+ ' (+ and a whitespace, such as + *.jpg). To specify an exclusion, start the filter pattern with '- ' (- and a whitespace, such as - *.png).

To set filters for a specific user, run the following asconfigurator command:

> asconfigurator -x "set_user_data;user_name,username;file_filters,|filter1[|filter2]"

To set filters for all users, run the following:

> asconfigurator -x "set_node_data;file_filters,|filter1[|filter2]"

The separator "|" is not required if only one filter is set.

Individual Matching Rules

Rules applied to directory and file names are limited regular expressions (referred to as "globs" in UNIX terminology). Rules use standard globbing syntax and globbing extensions.

Standard Globbing Syntax

Standard Globbing Description
/ The only recognized path separator.
\ Quotes any character literally, including itself. The \ character is exclusively a quoting operator, not a path separator.
* Matches zero or more characters, except a / or a . preceded immediately by a / character.
? Matches any single character, except a / or a . preceded immediately by a / character.
[ … ] Matches exactly one of a set of characters, except a / or a . preceded immediately by a / character.
[^… ] When ^ is the first character, matches exactly one character not in the set.
[!… ] When ! is the first character, matches exactly one character not in the set.
[x-x] Matches exactly one of a range of characters.
[:xxxxx:] For details about this type of wildcard, see any POSIX-standard guide to "globbing".

Examples of Standard Globbing

Standard Globbing Example Matches Does Not Match
/ abc/def/xyz abc/def/xyz abc/def
\ abc\? abc? abc\? abc/D abcD
* abc*f abcdef abc.def abc/f abc/def abc/.def
? abc?? abcde abcdef abc/d abc/.
[ … ] [abc]def adef bdef cdef abcdef ade
[^… ] [^abc]def zdef .def 2def adef bdef cdef /def /.def
[!… ] [!abc]def zdef .def 2def adef bdef cdef /def /.def
[x-x] [a-z]def cdef ydef Adef 2def .def

Globbing Extensions

Globbing Extensions Description
/** Like * except that it also matches the / character.
* or /** at end of rule Matches both directories and files.
/ at end of rule Matches directories only.
no / at end of rule Matches files only.
/ at start of rule Matches from the system's root directory (absolute path) only; that is, the entire string must be matched. Note: The / means the system's root, not the docroot, and not from the top level specified for the transfer set.

Examples of Globbing Extensions

Globbing Extensions Example Matches Does Not Match
/** abc/**/def abc/def abc/x/def abc/.wxy/def abc/wxy/tuv/def abc/xyz/def/ zabc/wxy/def
* at end of rule abc* abc/file abc/dir  
/** at end of rule abc/** abc/.file abc/dir abc/wxy/.dir abc/wxy/tuv/file abc/
/ at end of rule abc/*/ abc/dir abc/file
no / at end of rule file file dir
/ at start of rule /abc/def /abc/def .../abc/def

Rule Composition

Example Expected Behavior
--include A --include B
Matches all file or directory names that match A or B.
--include A --exclude B
Matches all file or directory names that match A; of the rest, excludes all that match B.
--exclude B --include A
Excludes all file or directory names that match B; of the rest, matches those that match A.
--exclude-from=FILE1 --include-from=FILE2
Read filter specifications from FILE1 and FILE2
  • Rules in the file are appended as if from a series of --include A --exclude B options
  • Lines that start with + or plus, space, (e.g., + A), are equivalent to the option --include A
  • Lines that start with - or minus, space (e.g., - B), are equivalent to the option --exclude B
  • Lines that start with .+ or dot, plus, space (e.g., .+ F), are equivalent to the option --include-from=F
  • Lines that start with .- or dot, minus, space (e.g., .+ G), are equivalent to the option --exclude-from=G
  • Lines that start with . or dot, space, name a file to read for more rules, but without a default (i.e., each line must start with one of the above commands).
  • Other lines default to include (for --include-from) or exclude (for --exclude-from) rules
  • Leading white space, blank lines and comment lines (# comment) are ignored

A file or directory name that does not match any rule is still tracked, as if by a final "+ *" and "+ .*". Also, to reliably exclude all unmatched files, add two final rules: "- *" and "- .*".

Semantics

Excluded new files are invisible to async. Files that have already been synced continue to be tracked even when they have, or get, a name that is now excluded. When run with --exclude FILE3:

Local event Effect on peer (previously synced) Clean start
mv FILE4 FILE3 mv FILE4 FILE3 rm FILE4
rm FILE3 rm FILE3 (ignored)
cp FILE4 FILE3 cp FILE4 FILE3 (ignored)
mv FILE3 FILE4 mv FILE3 FILE4 new file FILE4