Not for download from (S)FTP, no - at least, not by 'remembering' - at least, not
directly.
There is a mode called Fast Backup (actually, two FB methods: one uses a database to 'remember', the other mode uses the standard archive bit) but due to SB's inherent design* this can only be used on the same 'side' as any non-standard resource (S/FTP, mail server storage, Cloud...)
if a non-standard resource is involved. You therefore can't use it to remember which files on 'local' Destination have been downloaded already (whether still there or not)
if the Source is non-standard (so SFTP as Source is 'out' as a
single-step process that 'memorizes' stuff already downloaded).
You
can use Fast Backup to remember which files were transferred from a standard Source, i.e. disk or UNC path to a similar standard Destination. This means by extension (and a little lateral thinking) you could conceivably use a local buffer area which holds 'files from SFTP' (
all of them) as a mid-point. You could allow the matching presence (or not) of a file in this buffer area to govern whether an initial SFTP profile downloads a given file from the SFTP server or not (by default, new/different = yes, otherwise no), and then use a separate Fast Backup profile as 'stage-2' to remember which files have already been copied
from this buffer area to the 'real' location where files are transferred briefly for further processing (that is, the Destination in your question). So
- SFTP > Local Buffer Area (using a normal SFTP profile)
- Local Buffer Area > Destination (disk:disk or disk:LAN Fast Backup profile, remembering which files in buffer area have been copied to the for-processing Destination already & not doing so again)
Read the Help carefully on Fast Backup and also check the
Knowledge Base (do a search on 'fast') to familiarize yourself with the quirks and/or things that may not have occurred to you...)
Alternatively, depending on your circumstances, it might be possible to leverage an age-limit ('only process files stamped XXX or newer, or similar') via
Modify -> Expert -> Compare Options -> Date & Time
which can use a 'rolling period' (that is, 'within the last X') but you need to read the contextual Help carefully for details of the rounding used, and when it will trigger (or not) for a given file. You can use higher values of smaller units (up to a max of 65535) for finer-tuning - for example, you could use '1440 minutes' instead of '1' day' (but you can't use '86400 seconds', because 86400 exceeds 65535)
* I understand that to change this would require a complete re-write 'from ground up' and the amount of functionality that would need to be revised means this is almost certainly not going to happen
Statistics: Posted by cliffhanger — Mon Apr 20, 2015 3:26 pm