Include/Exclude Filtering Rules

async include/exclude matching rules, rule composition and semantics

Notes on Separators and Case

You can synchronize between a Linux endpoint and a Linux server, as well as between a Linux endpoint and a Windows server. In async, the path separator "/" works equally well on Windows and other platforms. The path separator "\" is platform-agnostic only for the options -d/r/L/R/B/b and --keep-dir-local/remote. In filtering rules, however, "\" is exclusively a quoting operator and "/" is the only path separator recognized. The examples below use "/" uniformly.
Case always matters, even if the scanned file system does not enforce such a distinction. For example, on Windows FAT / NTFS file systems and Mac OSX HPFS+, if the user writes "DEBUG", it matches files "Debug" and "debug". In filter rules, however, the comparison is always exact. To pick up both "Debug" and "debug", the pattern must be "[Dd]ebug".

Specifying Rules on the Command Line

Rules may be specified on the command line (for example, "--exclude PATTERN") or read from a file (for example, "--exclude-from=FILE") to make async skip or include specific directories and files. The following command-line options can be freely intermixed:

--exclude
--include
--exclude-from
--include-from

Rules are applied in the order that they are encountered, and the first matching rule (whether including or excluding) takes precedence. For example, if the file aspera-test specifies rules W, X and Y, then the following options apply rules V, W, X, Y, Z (in that order).

--include V --exclude-from aspera-test --include Z

Note:

Directories and files are visited in strict depth order, so that with the command-line options "--exclude /above/ --include /above/below the file /above/below is never considered. The directory /above/ is not scanned; thus, "/above/below" has no opportunity to be checked against any rules.

Individual Matching Rules

Rules applied to directory and file names are limited regular expressions (referred to as "globs" in UNIX terminology).

Standard Globbing, as in POSIX sh(1) and find(1)

* ? [abc] [a-z] *.[!cos] *.[^cos] [![:digit:][:blank:]]

"*" matches any number of characters
"?" matches any one character
"[]" matches exactly one of a set of characters.

"[]" may also contain the following:

"^" or "!", which--if it appears as the first character--specifies that the sequence will match any character NOT in the set.
Ranges, which are specified by giving the first and last characters with a "-" between them.
[:alnum:] — alphanumeric, same as [:alpha:][:digit:]
[:alpha:] — alphabetic
[:blank:] — space or tab
[:cntrl:] — control character
[:digit:] — digit, 0-9
[:graph:] — not [:blank:] and not [:cntrl:]
[:lower:] — lowercase alphabetic
[:punct:] — [:graph:] but not [:alnum:]
[:space:] — space, tab, linefeed, vertical tab, carriage return
[:upper:] — uppercase alphabetic
[:xdigit:] — 0-9a-fA-F

Additional Notes on Standard Globbing:

Directory names, file names and glob literal characters may be multibyte unicode. For example, voilà is matched by rules voilà, voil[àáâ], voil[[:alpha:]] (but not voil[[=a=]] or voil[[.a-grave.]]).
\ quotes any character literally, including itself. Example: pattern a\*b\[c[\]]d\?e\\f matches file name a*b[c]d?e\f.
Wildcards *, [] and ? do not match /, nor a . immediately after / (ie /.).
- abc/def is not matched by abc*def, abc[/]def or abc?def
- abc/.def is not matched by any of abc/*, abc/[![:alpha:]]def, or abc/?def
- abc/.def is matched by */.???

Globbing Extensions

Note: ** must be bounded by / literals, or end-of-string (e.g. /**/).

The wildcard /** is like *, but also matches /.

abc/**/def matches abc/wxy/def
abc/**/def matches abc/def
abc/**/def does not match abc/wxy/.def
abc/**/def does match abc/.wxy/def (only the last /. counts)

File-type specificity:

A rule ending with / matches only directories, whereas a rule ending without / matches only files.
A rule ending with * or /** matches matches both files and directories.

Rules that start with / are anchored (i.e., the entire string must be matched).

rule /abc/**/def matches /abc/wxy/def
rule abc/**/def matches xyz/abc/wxy/def
rule /abc/**/def does not match /xyz/abc/wxy/def

Rule Composition

Example	Expected Behavior
--include A --include B	Matches all file or directory names that match A or B.
--include A --exclude B	Matches all file or directory names that match A; of the rest, excludes all that match B.
--exclude B --include A	Excludes all file or directory names that match B; of the rest, matches those that match A.
--exclude-from=FILE1 --include-from=FILE2	Read filter specifications from FILE1 and FILE2 Rules in the file are appended as if from a series of --include A --exclude B options Lines that start with + or plus, space, (e.g., + A), are equivalent to the option --include A Lines that start with - or minus, space (e.g., - B), are equivalent to the option --exclude B Lines that start with .+ or dot, plus, space (e.g., .+ F), are equivalent to the option --include-from=F Lines that start with .- or dot, minus, space (e.g., .+ G), are equivalent to the option --exclude-from=G Lines that start with . or dot, space, name a file to read for more rules, but without a default (i.e., each line must start with one of the above commands). Other lines default to include (for --include-from) or exclude (for --exclude-from) rules Leading white space, blank lines and comment lines (# comment) are ignored

Note: A file or directory name that does not match any rule is still tracked, as if by a final "+ *" and "+ .*". Also, to reliably exclude all unmatched files, add two final rules: "- *" and "- .*".

Semantics

Excluded new files are invisible to async. Files that have been synced already continue to be tracked even when they have, or get, a name that is now excluded. When run with --exclude FILE3:

Local event	Effect on peer (previously synced)	Clean start
mv FILE4 FILE3	mv FILE4 FILE3	rm FILE4
rm FILE3	rm FILE3	(ignored)
cp FILE4 FILE3	cp FILE4 FILE3	(ignored)
mv FILE3 FILE4	mv FILE3 FILE4	new file FILE4