Sync Command Examples
- Continuous synchronization of a daily archive of large files on a Windows machine to
Linux machine, preserving Windows acls, run as an async pull on the
Linux
machine:
$ async -L /sync/logs -N backup -d /sync/backup -r alligator@everglades.company.com:"C:\data\" -i /.ssh/lion_private_key -K pull --remote-scan-interval=4h --preserve-acls=metafile --remote-preserve-acles=metafile -C --exclude-dirs-older-than=1w0d0h0m0s
Details:
- Logs are stored on the Linux machine in the specified location.
- The user, lion, authenticates with an SSH key using the -i
option
- Because the files in the backup are large, remote-scan-interval
is used to scan the Windows machine every 4 hours, which forces an additional scan
in case any notifications are missed.
- In order to preserve Windows acls in the backup, both
preserve-acls=metafile and
remote-preserve-acls=metafile must be specified.
- Since the archive directory creates a new directory for each day, use
exclude-dirs-older-than=1w0d0h0m0s to avoid scanning directories
that are no longer changing (older than a week).
- High performance push sync of many (10,000s) of small files (<10 KB) between Windows
machines:
> async -L c:/logs:200 -q -N small-files -c none --pending-max=10000 --preserve-acls=native --transfer-threads=4 -R c:/logs:200 -d c:/data/ -r bobcat@192.168.4.24:"C:\data\" -K push -l 500m
Details:
- Specifying the logging locations (-L and -R) is
optional. Adding :200 to the end of the log directory value allows
the logs to reach 200 MB before being rotated.
- If the connection is secure, disabling encryption using -c none
may boost performance.
- Increase the number of pending files from the default of 2000 using
--pending-max=10000.
- The --preserve-acls=native option preserves Windows ALCs.
- Using more FASP threads to move the data can improve performance, set with
--transfer-threads=4. The number of threads should not exceed the
number of CPU cores (the lower value of the client and server machines).
- The user must enter the password at the prompt because it is not provided in the
command. Aspera recommends using SSH keys for authentication, but this is not
required.
- Non-continuous bidirectional sync of directories containing a mix of large and small
files in which small files are synced using one thread and large files use another,
run on a Linux machine to a Mac OS X
machine:
$ async -L /sync/logs -q -N sync-2017-01-01 -d /images --user=gazelle@company.com --host=10.4.25.10 -r Library/data/images -i /lion/.ssh/lion_private_key -R Library/sync/logs --transfer-threads=2:100k -K bidi
Details:
- Logs are saved in the specified locations on both machines.
- The user authenicates with an SSH key using the -i option.
- The user and host are specified as separate options, rather than as part of the
destination folder, so that the username with an @ can be used (@ is reserved in an
-r argument for specifying the host).
- The sync uses two threads, one for files larger than 100 KB and one for files less
than or equal to 100 KB, specified with the --transfer-threads
option.
Sync Output Example
When Sync is run in interactive mode, the status of each file in the synchronized directory is
displayed in a list similar to the following:
/file1 SYNCHRONIZED
/file2 SYNCHRONIZED(exs)
/file3 SYNCHRONIZED(skp)
/file4 SYNCHRONIZED(del)
The status may be one of the following options:
- SYNCHRONIZED: file transferred
- SYNCHRONIZED(skp): file skipped
- SYNCHRONIZED(del): file deleted
- SYNCHRONIZED(ddp): dedup (duplicate files present)
- SYNCHRONIZED(exs): file exists
- SYNCHRONIZED(mov): file has changed (renamed, moved, or different attributes)