FTPauto is a simple, but highly advanced and configurable FTP-client wrap-around written in #Bash for Unix. It is based on lftp and helps to manage transfers.
- Send files easily with lftp (To and from FTP(s), and between serveres - FXP, and SFTP)
- Highly customizable command line optopns
- Monitor free space or server status on the remote server (pre/post check)
- Progressbar with estimated time of transferes
- History (Transfer logs, total transfered etc.)
- Queue manager for failed, queued transfere
- Packing/splitting directory/files into rar-files. Including sfv to verify files at end-server.
- Multiple user support
- Delay transfer to start at a specific time
- Support pre/post transfer using external scripts
- Sorting (regex-based or manually)
- Regex-based exclution of files or folders
- Seamless read-content from rar files using rar2fs fuse
- Send push notification to phones etc. with Pushover
- Automatic transfers with the use of FlexGet
If you find this tool helpful, a small donation is appreciated, 
- Requirements
- Installation (Recommended)
- Get svn
- Manual install
- Upgrading
- Configuration
- Adding user
- The config
- Usage
- Arguments
- Debugging
- 3rd party uses
- FlexGet
- Download methods
- Server download
- Clent RSS download
- Clent FTP download
- Scheduling
- Cron
- Daemon
- Multiple users
You should have User and sudo or root access to use and install, respectively. Script is mainly written to Debian/Ubuntu and is guaranteed to work under these!
There are several way to install FTPauto. Sudo is necesary to install some dependent tools
To get FTPauto, download and execute the installer: download The installer will set up the environment and will help to install the necessary programs. During the installation you will also be able to set up a user!
mkdir FTPauto && cd FTPauto
wget https://raw.github.com/Meliox/FTPauto/master/install.sh
bash install.sh installand update
bash install.sh updateFollow the instructions to set up a user and then you're ready to use FTPauto! If you skipped user setup or need help go to configuration to set up a user.
Alternatively, you can get it from Github (This version may contain unfinished features and be unstable).
git clone ssh://[email protected]/Meliox/FTPauto.gitRun
bash install.sh installand update
git pull
bash install.sh updateFirst thing that need to be done is to create a user and edit the users settings. The setting that is to be edited is shown in settings.
Add user:
bash ftpauto.sh --user=<USERNAME> --addThe users configuration must be edited prior use:
bash ftpauto.sh --edit --user=<USERNAME>Note: Editing can also be done manually after adding the user!
nano ~/users/<USERNAME>/configEditing the config should be straight forward, but if you have troubles more info can be found here The config. After this you may now start using FTPauto. See more usage.
The most important setting is "transferetype". Depending on the solution you can FTPauto can do
FTP
SERVER --> client : upftp(s) or upsftp
server <-- CLIENT : downftp(s)
FXP
SERVER <-> server : fxp
The capital letters state where FTPauto is executed and therefor giving the correct path is important. For FXP transferes ftphost1 is SERVER. For example, see example
The rest of the settings should be selfexplanatory. The config can be found here: https://github.com/Meliox/FTPauto/blob/master/dependencies/help.sh#L90.
After configuration transferes can be made as written below:
bash ftpauto.sh --user=<USERNAME> --path=~/something/Several arguments can be used, see here: arguments.
Can be shown with:
bash ftpauto.sh --helpHere's an overview as well
== Required ==
--user=<USER> | Required at all times
== Session manipulation ==
--pause | Terminates transfer and leaves queue intact
--start | Begins transfer from queue and let it finish queue. Only to be used for sessions!
--stop | Terminates transfer and remove queue and current id
== Item manipulation ==
--list | Lists all items in queue
= Required =
--id=<id> | Id for <PATH> you want to manipulate. Find them in the queuefile. See --list
= Options =
--clear | Remove everything in queue
--down | Move <ID> down
--forget | Remove <ID> from queue
--path=<PATH> | <PATH> used to transfer now!
--queue | Sends <PATH> to queue WITHOUT starting script if autostart=false in config.
NOTE that --path <ITEM> is required for this to work.
--source=<SOURCE> | Source is used to show how the download has been started. The
following is possible:
MANDL=manual download(if nothing is used)
WEBDL=download from webpage
FLXDL=autodownload from flexget
other can be used as well...
--up | Move <ID> Up
--sort | Sorts transfer into passed directory. Usage --sort=somedir/somedir2/
This will overwrite automatic sorting.
--sort=nosort transfers into ftpcomplete directory if sorting is enabled
== User manipulation ==
--add | Add user --add=<USER>
--edit | Edit <USER> config
--purge | Removes all user history and configs
--remove | Removes all user history
== Server ==
--freespace | Checks how much free space is available (slow if on remote server)
--online | Checks if server is online and writeable
== Optional ==
--bg | Transfer is done in background
--debug | Debugs to logfile
--delay | Delays transfer until X. Has to be in this format 01/01/2010 12:00 (Month/Day/Year 24h-time)
--exec_post | Execute commands after download
--exec_pre | Execute commands before download
--force | Transfer file despite something is running
--help | Print help info
--progress | While transferring, this will print out progress if enabled in config
--quiet | Suppresses all output
--test | Shows what transfer is going to happen
--verbose | Debugs to consoleIf the script for some reason should fail, it is easy to debug. Debugging can either be permanently set if the error comes and goes. This setting can be in ftpauto.sh by altering the line 3:
verbose="0" #0 Normal info | 1 debug console | 2 debug into logfileDebugging into logfile will create a ftpscript logfile and a ftp logfile, so that everything can be looked at later. Debugging to console only will show script.
Debugging may also be used as an argument, see Arguments.
FTPauto can be used in combination with other software. Here are a few examples listed.
FlexGet is a multi-purpose automation tool for content like torrents, nzbs, podcasts, comics, series, movies, etc. It can use different kinds of sources like RSS-feeds, html pages, csv files, search engines and there are even plugins for sites that do not provide any kind of useful feeds. Here we will use FlexGet to scan directories and let Flexget send the approved files. First prerequisite is to install Flexget, which can be found here FlexGet#Install. Then an appropiate config has to be written as the following examples. More info on FlexGet and how it work is not going to be explained as it is done so very nicely on their homepage, FlexGet#Configuration
There are a few ways of downloading with the use of Flexget, these are explained in the subsections below.
One important thing to remember is to check if the config is writtten properly. Check it by
$ bin/flexget --test check
2014-01-10 11:15 INFO check Config passed check.And it will show if the config passes or fails.
This is a serverside configuration, meaning the server that has the files transfers the files.
tasks:
download:
listdir: [~/TV/, ~/path2/]
series:
720p:
- TVSHOW1
- TVSHOW2
exec:
fail_entries: yes
allow_background: yes
auto_escape: yes
on_output:
for_accepted: 'sleep 5; bash ~/ftpauto.sh --path="{{location}}/" --user=<USER> --source=FLXDL &'This is a clientside configuration, meaning that the client looks for new files in order to transfer them.
tasks:
download:
inputs:
- rss:
url: http://some.url.rss
series:
720p:
- TVSHOW1
- TVSHOW2
manipulate:
# Flexget can manipulate the url. If the url is made by flexget it usually has the original path written first.
# This can before with this
- url:
replace:
regexp: 'file://<path before ftp path>/'
format: '/'
exec:
fail_entries: yes
allow_background: yes
auto_escape: yes
on_output:
for_accepted: 'sleep 5; bash ~/ftpauto.sh --path="{{location}}/" --user=<USER> --source=FLXDL &'This is a clientside configuration, meaning that the client looks for new files in order to transfer them
tasks:
download:
inputs:
- ftp_list:
config:
use-ssl: <yes/no>
name: <ftp name>
username: <username>
password: <password>
host: <host to connect>
port: <port>
dirs:
- <directory 1>
- <directory 2>
series:
# will download series with name and quality
720p:
- <Tv show 1>
- <Tv show 2>
regexp:
# will download anything that matches the giving regex
accept:
- <regexp>:
from:
- title
# will not download matches of the giving regex
accept:
- <regexp>:
from:
- title
manipulate:
- url:
extract: (?:\:\d+\/)(.*)
exec:
fail_entries: yes
allow_background: yes
auto_escape: yes
on_output:
for_accepted: 'sleep 5; bash ~/ftpauto.sh --path="{{url}}/" --user=<USER> --source=FLXDL &'Although flexget supports download from ftp also, FTPauto makes it possible to resume and sort downloads, also using the queue system and finally FTPauto can be used without Flexget.
Flexget can be scheduled to run at specific times two different ways. Remember to make sure that your config passes check.
Add it to crontab and it may be added to crontab. Do this by and add this entry:
$ crontab -e
# write
*/5 * * * * ~/bin/flexget --cron(Where 5 minutes is the interval checking for new files)
Flexget also support daemon mode, which means that in can be run in the background, periodically running tasks on a schedule, or running the tasks initiated by another instance of FlexGet.
The following should then be added to your config
schedules:
- tasks: [list, of, tasks or * for all]
interval:
<weeks|days|hours|minutes>: <#>
on_day: <monday|tuesday...>
at_time: HH:MM [am|pm] # 24h time can also be usedMore information here Flexget-daemon
Flexget only handle one user PER show, so if several users see the same you need several configs. An addition argument then has to be used with the cron, daemon or check option
$ /bin/flexget -c ~<config path>