Those days we almost use cloud for everthing. But sometimes we need to pull files from sftp server. Here are two solutions for that
Pull and remove with sftp
This solution pulls the files then removes them from the remote. There is a gotcha that if you expect a lot of files there might be a chance a file to arrive while the “get -r …” command is executing. Then the “rm *” will remove it. So this is suitable if you expect a few files a week/day
Create a batchfile.sh
get -r upload/* incoming/
rm upload/*Then add cron
0 5 * * * /usr/bin/sftp -b batchfile.sh username@sftp-corp.company.comOnly pulling with lftp
When I don’t have permissions to remove the files from the remote sftp I use the following off-the-shelf aproach.
This cron is synchronizing files all files to /home/USERNAME/incoming
0 5 * * * /usr/bin/lftp -u USERNAME,none -e 'mirror --newer-than="now-7days" --only-newer --exclude .ssh --only-missing / /home/USERNAME/incoming; quit' sftp://sftp-corp.company.com