Merge pull request #9112 from freqtrade/new_release

New release 2023.8

.github/dependabot.yml

@@ -10,7 +10,7 @@ updates:
  directory: "/"
  schedule:
  interval: weekly
- open-pull-requests-limit: 10
+ open-pull-requests-limit: 15
  target-branch: develop
 
 - package-ecosystem: "github-actions"

.github/workflows/ci.yml

@@ -461,15 +461,15 @@ jobs:
  python setup.py sdist bdist_wheel
 
  - name: Publish to PyPI (Test)
- uses: pypa/[email protected].8
+ uses: pypa/[email protected].10
  if: (github.event_name == 'release')
  with:
  user: __token__
  password: ${{ secrets.pypi_test_password }}
  repository_url: https://test.pypi.org/legacy/
 
  - name: Publish to PyPI
- uses: pypa/[email protected].8
+ uses: pypa/[email protected].10
  if: (github.event_name == 'release')
  with:
  user: __token__

.pre-commit-config.yaml

@@ -8,7 +8,7 @@ repos:
  # stages: [push]
 
  - repo: https://github.com/pre-commit/mirrors-mypy
- rev: "v1.3.0"
+ rev: "v1.5.0"
  hooks:
  - id: mypy
  exclude: build_helpers
@@ -18,7 +18,7 @@ repos:
  - types-requests==2.31.0.2
  - types-tabulate==0.9.0.3
  - types-python-dateutil==2.8.19.14
- - SQLAlchemy==2.0.19
+ - SQLAlchemy==2.0.20
  # stages: [push]
 
  - repo: https://github.com/pycqa/isort

.readthedocs.yml

@@ -1,8 +1,14 @@
 # .readthedocs.yml
+version: 2
 
 build:
- image: latest
+ os: "ubuntu-22.04"
+ tools:
+ python: "3.11"
 
 python:
- version: 3.8
- setup_py_install: false
+ install:
+ - requirements: docs/requirements-docs.txt
+
+mkdocs:
+ configuration: mkdocs.yml

build_helpers/install_ta-lib.sh

@@ -8,8 +8,9 @@ if [ -n "$2" ] || [ ! -f "${INSTALL_LOC}/lib/libta_lib.a" ]; then
  tar zxvf ta-lib-0.4.0-src.tar.gz
  cd ta-lib \
  && sed -i.bak "s|0.00000001|0.000000000000000001 |g" src/ta_func/ta_utility.h \
- && curl 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.guess' -o config.guess \
- && curl 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.sub' -o config.sub \
+ && echo "Downloading gcc config.guess and config.sub" \
+ && curl -s 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.guess' -o config.guess \
+ && curl -s 'https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.sub' -o config.sub \
  && ./configure --prefix=${INSTALL_LOC}/ \
  && make
  if [ $? -ne 0 ]; then

build_helpers/install_windows.ps1

@@ -5,7 +5,7 @@ python -m pip install --upgrade pip wheel
 $pyv = python -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')"
 
 
-pip install --find-links=build_helpers\ TA-Lib
+pip install --find-links=build_helpers\ --prefer-binary TA-Lib
 
 pip install -r requirements-dev.txt
 pip install -e .

config_examples/config_full.example.json

@@ -89,7 +89,6 @@
  ],
  "exchange": {
  "name": "binance",
- "sandbox": false,
  "key": "your_exchange_key",
  "secret": "your_exchange_secret",
  "password": "",
@@ -206,6 +205,6 @@
  "recursive_strategy_search": false,
  "add_config_files": [],
  "reduce_df_footprint": false,
- "dataformat_ohlcv": "json",
- "dataformat_trades": "jsongz"
+ "dataformat_ohlcv": "feather",
+ "dataformat_trades": "feather"
 }

docs/bot-basics.md

@@ -7,7 +7,7 @@ This page provides you some basic concepts on how Freqtrade works and operates.
 * **Strategy**: Your trading strategy, telling the bot what to do.
 * **Trade**: Open position.
 * **Open Order**: Order which is currently placed on the exchange, and is not yet complete.
-* **Pair**: Tradable pair, usually in the format of Base/Quote (e.g. XRP/USDT).
+* **Pair**: Tradable pair, usually in the format of Base/Quote (e.g. `XRP/USDT` for spot, `XRP/USDT:USDT` for futures).
 * **Timeframe**: Candle length to use (e.g. `"5m"`, `"1h"`, ...).
 * **Indicators**: Technical indicators (SMA, EMA, RSI, ...).
 * **Limit order**: Limit orders which execute at the defined limit price or better.
@@ -20,6 +20,20 @@ This page provides you some basic concepts on how Freqtrade works and operates.
 
 All profit calculations of Freqtrade include fees. For Backtesting / Hyperopt / Dry-run modes, the exchange default fee is used (lowest tier on the exchange). For live operations, fees are used as applied by the exchange (this includes BNB rebates etc.).
 
+## Pair naming
+
+Freqtrade follows the [ccxt naming convention](https://docs.ccxt.com/#/README?id=consistency-of-base-and-quote-currencies) for currencies.
+Using the wrong naming convention in the wrong market will usually result in the bot not recognizing the pair, usually resulting in errors like "this pair is not available".
+
+### Spot pair naming
+
+For spot pairs, naming will be `base/quote` (e.g. `ETH/USDT`).
+
+### Futures pair naming
+
+For futures pairs, naming will be `base/quote:settle` (e.g. `ETH/USDT:USDT`).
+
+
 ## Bot execution logic
 
 Starting freqtrade in dry-run or live mode (using `freqtrade trade`) will start the bot and start the bot iteration loop.

docs/bot-usage.md

@@ -3,7 +3,7 @@
 This page explains the different parameters of the bot and how to run it.
 
 !!! Note
- If you've used `setup.sh`, don't forget to activate your virtual environment (`source .env/bin/activate`) before running freqtrade commands.
+ If you've used `setup.sh`, don't forget to activate your virtual environment (`source .venv/bin/activate`) before running freqtrade commands.
 
 !!! Warning "Up-to-date clock"
  The clock on the system running the bot must be accurate, synchronized to a NTP server frequently enough to avoid problems with communication to the exchanges.

docs/configuration.md

@@ -188,7 +188,6 @@ Mandatory parameters are marked as **Required**, which means that they are requi
 | `max_entry_position_adjustment` | Maximum additional order(s) for each open trade on top of the first entry Order. Set it to `-1` for unlimited additional orders. [More information here](strategy-callbacks.md#adjust-trade-position). <br> [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `-1`.*<br> **Datatype:** Positive Integer or -1
 | | **Exchange**
 | `exchange.name` | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename). <br> **Datatype:** String
-| `exchange.sandbox` | Use the 'sandbox' version of the exchange, where the exchange provides a sandbox for risk-free integration. See [here](sandbox-testing.md) in more details.<br> **Datatype:** Boolean
 | `exchange.key` | API key to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
 | `exchange.secret` | API secret to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
 | `exchange.password` | API password to use for the exchange. Only required when you are in production mode and for exchanges that use password for API requests.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String
@@ -251,8 +250,8 @@ Mandatory parameters are marked as **Required**, which means that they are requi
 | `db_url` | Declares database URL to use. NOTE: This defaults to `sqlite:///tradesv3.dryrun.sqlite` if `dry_run` is `true`, and to `sqlite:///tradesv3.sqlite` for production instances. <br> **Datatype:** String, SQLAlchemy connect string
 | `logfile` | Specifies logfile name. Uses a rolling strategy for log file rotation for 10 files with the 1MB limit per file. <br> **Datatype:** String
 | `add_config_files` | Additional config files. These files will be loaded and merged with the current config file. The files are resolved relative to the initial file.<br> *Defaults to `[]`*. <br> **Datatype:** List of strings
-| `dataformat_ohlcv` | Data format to use to store historical candle (OHLCV) data. <br> *Defaults to `json`*. <br> **Datatype:** String
-| `dataformat_trades` | Data format to use to store historical trades data. <br> *Defaults to `jsongz`*. <br> **Datatype:** String
+| `dataformat_ohlcv` | Data format to use to store historical candle (OHLCV) data. <br> *Defaults to `feather`*. <br> **Datatype:** String
+| `dataformat_trades` | Data format to use to store historical trades data. <br> *Defaults to `feather`*. <br> **Datatype:** String
 | `reduce_df_footprint` | Recast all numeric columns to float32/int32, with the objective of reducing ram/disk usage (and decreasing train/inference timing in FreqAI). (Currently only affects FreqAI use-cases) <br> **Datatype:** Boolean. <br> Default: `False`.
 
 ### Parameters in the strategy

docs/data-analysis.md

@@ -27,7 +27,7 @@ For this to work, first activate your virtual environment and run the following
 
 ``` bash
 # Activate virtual environment
-source .env/bin/activate
+source .venv/bin/activate
 
 pip install ipykernel
 ipython kernel install --user --name=freqtrade

docs/data-download.md

@@ -27,11 +27,11 @@ usage: freqtrade download-data [-h] [-v] [--logfile FILE] [-V] [-c PATH]
  [--exchange EXCHANGE]
  [-t TIMEFRAMES [TIMEFRAMES ...]] [--erase]
  [--data-format-ohlcv {json,jsongz,hdf5,feather,parquet}]
- [--data-format-trades {json,jsongz,hdf5}]
+ [--data-format-trades {json,jsongz,hdf5,feather}]
  [--trading-mode {spot,margin,futures}]
  [--prepend]
 
-optional arguments:
+options:
  -h, --help show this help message and exit
  -p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
  Limit command to these pairs. Pairs are space-
@@ -48,26 +48,26 @@ optional arguments:
  --dl-trades Download trades instead of OHLCV data. The bot will
  resample trades to the desired timeframe as specified
  as --timeframes/-t.
- --exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
- config is provided.
+ --exchange EXCHANGE Exchange name. Only valid if no config is provided.
  -t TIMEFRAMES [TIMEFRAMES ...], --timeframes TIMEFRAMES [TIMEFRAMES ...]
  Specify which tickers to download. Space-separated
  list. Default: `1m 5m`.
  --erase Clean all existing data for the selected
  exchange/pairs/timeframes.
  --data-format-ohlcv {json,jsongz,hdf5,feather,parquet}
  Storage format for downloaded candle (OHLCV) data.
- (default: `json`).
- --data-format-trades {json,jsongz,hdf5}
+ (default: `feather`).
+ --data-format-trades {json,jsongz,hdf5,feather}
  Storage format for downloaded trades data. (default:
- `jsongz`).
+ `feather`).
  --trading-mode {spot,margin,futures}, --tradingmode {spot,margin,futures}
  Select Trading mode
  --prepend Allow data prepending. (Data-appending is disabled)
 
 Common arguments:
  -v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
- --logfile FILE Log to the file specified. Special values are:
+ --logfile FILE, --log-file FILE
+ Log to the file specified. Special values are:
  'syslog', 'journald'. See the documentation for more
  details.
  -V, --version show program's version number and exit
@@ -157,7 +157,7 @@ Freqtrade currently supports the following data-formats:
 * `json` - plain "text" json files
 * `jsongz` - a gzip-zipped version of json files
 * `hdf5` - a high performance datastore
-* `feather` - a dataformat based on Apache Arrow (OHLCV only)
+* `feather` - a dataformat based on Apache Arrow
 * `parquet` - columnar datastore (OHLCV only)
 
 By default, OHLCV data is stored as `json` data, while trades data is stored as `jsongz` data.
@@ -255,7 +255,7 @@ usage: freqtrade convert-data [-h] [-v] [--logfile FILE] [-V] [-c PATH]
  [--trading-mode {spot,margin,futures}]
  [--candle-types {spot,futures,mark,index,premiumIndex,funding_rate} [{spot,futures,mark,index,premiumIndex,funding_rate} ...]]
 
-optional arguments:
+options:
  -h, --help show this help message and exit
  -p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
  Limit command to these pairs. Pairs are space-
@@ -266,19 +266,20 @@ optional arguments:
  Destination format for data conversion.
  --erase Clean all existing data for the selected
  exchange/pairs/timeframes.
- --exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
- config is provided.
+ --exchange EXCHANGE Exchange name. Only valid if no config is provided.
  -t TIMEFRAMES [TIMEFRAMES ...], --timeframes TIMEFRAMES [TIMEFRAMES ...]
  Specify which tickers to download. Space-separated
  list. Default: `1m 5m`.
  --trading-mode {spot,margin,futures}, --tradingmode {spot,margin,futures}
  Select Trading mode
  --candle-types {spot,futures,mark,index,premiumIndex,funding_rate} [{spot,futures,mark,index,premiumIndex,funding_rate} ...]
- Select candle type to use
+ Select candle type to convert. Defaults to all
+ available types.
 
 Common arguments:
  -v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
- --logfile FILE Log to the file specified. Special values are:
+ --logfile FILE, --log-file FILE
+ Log to the file specified. Special values are:
  'syslog', 'journald'. See the documentation for more
  details.
  -V, --version show program's version number and exit
@@ -291,7 +292,6 @@ Common arguments:
  Path to directory with historical backtesting data.
  --userdir PATH, --user-data-dir PATH
  Path to userdata directory.
-
 ```
 
 ### Example converting data
@@ -314,7 +314,7 @@ usage: freqtrade convert-trade-data [-h] [-v] [--logfile FILE] [-V] [-c PATH]
  {json,jsongz,hdf5,feather,parquet}
  [--erase] [--exchange EXCHANGE]
 
-optional arguments:
+options:
  -h, --help show this help message and exit
  -p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
  Limit command to these pairs. Pairs are space-
@@ -325,12 +325,12 @@ optional arguments:
  Destination format for data conversion.
  --erase Clean all existing data for the selected
  exchange/pairs/timeframes.
- --exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
- config is provided.
+ --exchange EXCHANGE Exchange name. Only valid if no config is provided.
 
 Common arguments:
  -v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
- --logfile FILE Log to the file specified. Special values are:
+ --logfile FILE, --log-file FILE
+ Log to the file specified. Special values are:
  'syslog', 'journald'. See the documentation for more
  details.
  -V, --version show program's version number and exit
@@ -367,28 +367,28 @@ usage: freqtrade trades-to-ohlcv [-h] [-v] [--logfile FILE] [-V] [-c PATH]
  [-t TIMEFRAMES [TIMEFRAMES ...]]
  [--exchange EXCHANGE]
  [--data-format-ohlcv {json,jsongz,hdf5,feather,parquet}]
- [--data-format-trades {json,jsongz,hdf5}]
+ [--data-format-trades {json,jsongz,hdf5,feather}]
 
-optional arguments:
+options:
  -h, --help show this help message and exit
  -p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
  Limit command to these pairs. Pairs are space-
  separated.
  -t TIMEFRAMES [TIMEFRAMES ...], --timeframes TIMEFRAMES [TIMEFRAMES ...]
  Specify which tickers to download. Space-separated
  list. Default: `1m 5m`.
- --exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
- config is provided.
+ --exchange EXCHANGE Exchange name. Only valid if no config is provided.
  --data-format-ohlcv {json,jsongz,hdf5,feather,parquet}
  Storage format for downloaded candle (OHLCV) data.
- (default: `json`).
- --data-format-trades {json,jsongz,hdf5}
+ (default: `feather`).
+ --data-format-trades {json,jsongz,hdf5,feather}
  Storage format for downloaded trades data. (default:
- `jsongz`).
+ `feather`).
 
 Common arguments:
  -v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
- --logfile FILE Log to the file specified. Special values are:
+ --logfile FILE, --log-file FILE
+ Log to the file specified. Special values are:
  'syslog', 'journald'. See the documentation for more
  details.
  -V, --version show program's version number and exit
@@ -422,13 +422,12 @@ usage: freqtrade list-data [-h] [-v] [--logfile FILE] [-V] [-c PATH] [-d PATH]
  [--trading-mode {spot,margin,futures}]
  [--show-timerange]
 
-optional arguments:
+options:
  -h, --help show this help message and exit
- --exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
- config is provided.
+ --exchange EXCHANGE Exchange name. Only valid if no config is provided.
  --data-format-ohlcv {json,jsongz,hdf5,feather,parquet}
  Storage format for downloaded candle (OHLCV) data.
- (default: `json`).
+ (default: `feather`).
  -p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
  Limit command to these pairs. Pairs are space-
  separated.
@@ -439,7 +438,8 @@ optional arguments:
 
 Common arguments:
  -v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
- --logfile FILE Log to the file specified. Special values are:
+ --logfile FILE, --log-file FILE
+ Log to the file specified. Special values are:
  'syslog', 'journald'. See the documentation for more
  details.
  -V, --version show program's version number and exit
@@ -474,7 +474,7 @@ ETH/USDT 5m, 15m, 30m, 1h, 2h, 4h
 By default, `download-data` sub-command downloads Candles (OHLCV) data. Some exchanges also provide historic trade-data via their API.
 This data can be useful if you need many different timeframes, since it is only downloaded once, and then resampled locally to the desired timeframes.
 
-Since this data is large by default, the files use gzip by default. They are stored in your data-directory with the naming convention of `<pair>-trades.json.gz` (`ETH_BTC-trades.json.gz`). Incremental mode is also supported, as for historic OHLCV data, so downloading the data once per week with `--days 8` will create an incremental data-repository.
+Since this data is large by default, the files use the feather fileformat by default. They are stored in your data-directory with the naming convention of `<pair>-trades.feather` (`ETH_BTC-trades.feather`). Incremental mode is also supported, as for historic OHLCV data, so downloading the data once per week with `--days 8` will create an incremental data-repository.
 
 To use this mode, simply add `--dl-trades` to your call. This will swap the download method to download trades, and resamples the data locally.