Merge pull request #8978 from freqtrade/new_release

New Release 2023.7

.github/workflows/ci.yml

@@ -461,15 +461,15 @@ jobs:
  python setup.py sdist bdist_wheel
 
  - name: Publish to PyPI (Test)
- uses: pypa/[email protected].7
+ uses: pypa/[email protected].8
  if: (github.event_name == 'release')
  with:
  user: __token__
  password: ${{ secrets.pypi_test_password }}
  repository_url: https://test.pypi.org/legacy/
 
  - name: Publish to PyPI
- uses: pypa/[email protected].7
+ uses: pypa/[email protected].8
  if: (github.event_name == 'release')
  with:
  user: __token__

.pre-commit-config.yaml

@@ -13,12 +13,12 @@ repos:
  - id: mypy
  exclude: build_helpers
  additional_dependencies:
- - types-cachetools==5.3.0.5
+ - types-cachetools==5.3.0.6
  - types-filelock==3.2.7
- - types-requests==2.31.0.1
- - types-tabulate==0.9.0.2
- - types-python-dateutil==2.8.19.13
- - SQLAlchemy==2.0.17
+ - types-requests==2.31.0.2
+ - types-tabulate==0.9.0.3
+ - types-python-dateutil==2.8.19.14
+ - SQLAlchemy==2.0.19
  # stages: [push]
 
  - repo: https://github.com/pycqa/isort

build_helpers/install_windows.ps1

@@ -1,21 +1,11 @@
-# Downloads don't work automatically, since the URL is regenerated via javascript.
-# Downloaded from https://www.lfd.uci.edu/~gohlke/pythonlibs/#ta-lib
+# vendored Wheels compiled via https://github.com/xmatthias/ta-lib-python/tree/ta_bundled_040
 
 python -m pip install --upgrade pip wheel
 
 $pyv = python -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')"
 
-if ($pyv -eq '3.8') {
- pip install build_helpers\TA_Lib-0.4.26-cp38-cp38-win_amd64.whl
-}
-if ($pyv -eq '3.9') {
- pip install build_helpers\TA_Lib-0.4.26-cp39-cp39-win_amd64.whl
-}
-if ($pyv -eq '3.10') {
- pip install build_helpers\TA_Lib-0.4.26-cp310-cp310-win_amd64.whl
-}
-if ($pyv -eq '3.11') {
- pip install build_helpers\TA_Lib-0.4.26-cp311-cp311-win_amd64.whl
-}
+
+pip install --find-links=build_helpers\ TA-Lib
+
 pip install -r requirements-dev.txt
 pip install -e .

docker/docker-compose-freqai.yml

@@ -32,5 +32,5 @@ services:
  --logfile /freqtrade/user_data/logs/freqtrade.log
  --db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
  --config /freqtrade/user_data/config.json
- --freqai-model XGBoostClassifier
- --strategy SampleStrategy
+ --freqaimodel XGBoostRegressor
+ --strategy FreqaiExampleStrategy

docs/advanced-backtesting.md

@@ -103,6 +103,22 @@ The indicators have to be present in your strategy's main DataFrame (either for
 timeframe or for informative timeframes) otherwise they will simply be ignored in the script
 output.
 
+There are a range of candle and trade-related fields that are included in the analysis so are 
+automatically accessible by including them on the indicator-list, and these include:
+
+- **open_date :** trade open datetime
+- **close_date :** trade close datetime
+- **min_rate :** minimum price seen throughout the position
+- **max_rate :** maxiumum price seen throughout the position
+- **open :** signal candle open price
+- **close :** signal candle close price
+- **high :** signal candle high price
+- **low :** signal candle low price
+- **volume :** signal candle volumne
+- **profit_ratio :** trade profit ratio
+- **profit_abs :** absolute profit return of the trade 
+
+
 ### Filtering the trade output by date
 
 To show only trades between dates within your backtested timerange, supply the usual `timerange` option in `YYYYMMDD-[YYYYMMDD]` format:

docs/backtesting.md

@@ -305,7 +305,7 @@ A backtesting result will look like that:
 | Sharpe | 2.97 |
 | Calmar | 6.29 |
 | Profit factor | 1.11 |
-| Expectancy  | -0.15   |
+| Expectancy (Ratio) | -0.15 (-0.05) |
 | Avg. stake amount | 0.001 BTC |
 | Total trade volume | 0.429 BTC |
 | | |
@@ -324,6 +324,7 @@ A backtesting result will look like that:
 | Days win/draw/lose | 12 / 82 / 25 |
 | Avg. Duration Winners | 4:23:00 |
 | Avg. Duration Loser | 6:55:00 |
+| Max Consecutive Wins / Loss | 3 / 4 |
 | Rejected Entry signals | 3089 |
 | Entry/Exit Timeouts | 0 / 0 |
 | Canceled Trade Entries | 34 |
@@ -409,7 +410,7 @@ It contains some useful key metrics about performance of your strategy on backte
 | Sharpe | 2.97 |
 | Calmar | 6.29 |
 | Profit factor | 1.11 |
-| Expectancy  | -0.15   |
+| Expectancy (Ratio) | -0.15 (-0.05) |
 | Avg. stake amount | 0.001 BTC |
 | Total trade volume | 0.429 BTC |
 | | |
@@ -428,6 +429,7 @@ It contains some useful key metrics about performance of your strategy on backte
 | Days win/draw/lose | 12 / 82 / 25 |
 | Avg. Duration Winners | 4:23:00 |
 | Avg. Duration Loser | 6:55:00 |
+| Max Consecutive Wins / Loss | 3 / 4 |
 | Rejected Entry signals | 3089 |
 | Entry/Exit Timeouts | 0 / 0 |
 | Canceled Trade Entries | 34 |
@@ -467,6 +469,7 @@ It contains some useful key metrics about performance of your strategy on backte
 - `Best day` / `Worst day`: Best and worst day based on daily profit.
 - `Days win/draw/lose`: Winning / Losing days (draws are usually days without closed trade).
 - `Avg. Duration Winners` / `Avg. Duration Loser`: Average durations for winning and losing trades.
+- `Max Consecutive Wins / Loss`: Maximum consecutive wins/losses in a row.
 - `Rejected Entry signals`: Trade entry signals that could not be acted upon due to `max_open_trades` being reached.
 - `Entry/Exit Timeouts`: Entry/exit orders which did not fill (only applicable if custom pricing is used).
 - `Canceled Trade Entries`: Number of trades that have been canceled by user request via `adjust_entry_price`.
@@ -534,6 +537,7 @@ Since backtesting lacks some detailed information about what happens within a ca
 - ROI
  - exits are compared to high - but the ROI value is used (e.g. ROI = 2%, high=5% - so the exit will be at 2%)
  - exits are never "below the candle", so a ROI of 2% may result in a exit at 2.4% if low was at 2.4% profit
+ - ROI entries which came into effect on the triggering candle (e.g. `120: 0.02` for 1h candles, from `60: 0.05`) will use the candle's open as exit rate
  - Force-exits caused by `<N>=-1` ROI entries use low as exit value, unless N falls on the candle open (e.g. `120: -1` for 1h candles)
 - Stoploss exits happen exactly at stoploss price, even if low was lower, but the loss will be `2 * fees` higher than the stoploss price
 - Stoploss is evaluated before ROI within one candle. So you can often see more trades with the `stoploss` exit reason comparing to the results obtained with the same strategy in the Dry Run/Live Trade modes

docs/configuration.md

@@ -682,16 +682,14 @@ To use a proxy for exchange connections - you will have to define the proxies as
 { 
  "exchange": {
  "ccxt_config": {
- "aiohttp_proxy": "http://addr:port",
- "proxies": {
- "http": "http://addr:port",
- "https": "http://addr:port"
- },
+ "httpsProxy": "http://addr:port",
  }
  }
 }
 ```
 
+For more information on available proxy types, please consult the [ccxt proxy documentation](https://docs.ccxt.com/#/README?id=proxy).
+
 ## Next step
 
 Now you have configured your config.json, the next step is to [start your bot](bot-usage.md).

docs/developer.md

@@ -453,7 +453,13 @@ Once the PR against stable is merged (best right after merging):
 * Use the button "Draft a new release" in the Github UI (subsection releases).
 * Use the version-number specified as tag.
 * Use "stable" as reference (this step comes after the above PR is merged).
-* Use the above changelog as release comment (as codeblock)
+* Use the above changelog as release comment (as codeblock).
+* Use the below snippet for the new release
+
+??? Tip "Release template"
+ ````
+ --8<-- "includes/release_template.md"
+ ````
 
 ## Releases
 

docs/exchanges.md

@@ -259,10 +259,17 @@ The configuration parameter `exchange.unknown_fee_rate` can be used to specify t
 
 Futures trading on bybit is currently supported for USDT markets, and will use isolated futures mode.
 Users with unified accounts (there's no way back) can create a Sub-account which will start as "non-unified", and can therefore use isolated futures.
-On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that changes to this setting may result in exceptions and errors.
+On startup, freqtrade will set the position mode to "One-way Mode" for the whole (sub)account. This avoids making this call over and over again (slowing down bot operations), but means that changes to this setting may result in exceptions and errors
 
 As bybit doesn't provide funding rate history, the dry-run calculation is used for live trades as well.
 
+API Keys for live futures trading (Subaccount on non-unified) must have the following permissions:
+* Read-write
+* Contract - Orders
+* Contract - Positions
+
+We do strongly recommend to limit all API keys to the IP you're going to use it from.
+
 !!! Tip "Stoploss on Exchange"
  Bybit (futures only) supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
  On futures, Bybit supports both `stop-limit` as well as `stop-market` orders. You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.

docs/freqai-feature-engineering.md

@@ -261,7 +261,7 @@ class MyFreqaiModel(BaseRegressionModel):
  """
  feature_pipeline = Pipeline([
  ('qt', SKLearnWrapper(QuantileTransformer(output_distribution='normal'))),
- ('di', ds.DissimilarityIndex(di_threshold=1)
+ ('di', ds.DissimilarityIndex(di_threshold=1))
  ])
 
  return feature_pipeline

docs/freqai-parameter-table.md

@@ -42,7 +42,6 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
 | `use_SVM_to_remove_outliers` | Train a support vector machine to detect and remove outliers from the training dataset, as well as from incoming data points. See details about how it works [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm). <br> **Datatype:** Boolean.
 | `svm_params` | All parameters available in Sklearn's `SGDOneClassSVM()`. See details about some select parameters [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm). <br> **Datatype:** Dictionary.
 | `use_DBSCAN_to_remove_outliers` | Cluster data using the DBSCAN algorithm to identify and remove outliers from training and prediction data. See details about how it works [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan). <br> **Datatype:** Boolean. 
-| `inlier_metric_window` | If set, FreqAI adds an `inlier_metric` to the training feature set and set the lookback to be the `inlier_metric_window`, i.e., the number of previous time points to compare the current candle to. Details of how the `inlier_metric` is computed can be found [here](freqai-feature-engineering.md#inlier-metric). <br> **Datatype:** Integer. <br> Default: `0`.
 | `noise_standard_deviation` | If set, FreqAI adds noise to the training features with the aim of preventing overfitting. FreqAI generates random deviates from a gaussian distribution with a standard deviation of `noise_standard_deviation` and adds them to all data points. `noise_standard_deviation` should be kept relative to the normalized space, i.e., between -1 and 1. In other words, since data in FreqAI is always normalized to be between -1 and 1, `noise_standard_deviation: 0.05` would result in 32% of the data being randomly increased/decreased by more than 2.5% (i.e., the percent of data falling within the first standard deviation). <br> **Datatype:** Integer. <br> Default: `0`.
 | `outlier_protection_percentage` | Enable to prevent outlier detection methods from discarding too much data. If more than `outlier_protection_percentage` % of points are detected as outliers by the SVM or DBSCAN, FreqAI will log a warning message and ignore outlier detection, i.e., the original dataset will be kept intact. If the outlier protection is triggered, no predictions will be made based on the training dataset. <br> **Datatype:** Float. <br> Default: `30`.
 | `reverse_train_test_order` | Split the feature dataset (see below) and use the latest data split for training and test on historical split of the data. This allows the model to be trained up to the most recent data point, while avoiding overfitting. However, you should be careful to understand the unorthodox nature of this parameter before employing it. <br> **Datatype:** Boolean. <br> Default: `False` (no reversal).

docs/includes/pairlists.md

@@ -184,6 +184,8 @@ The RemotePairList is defined in the pairlists section of the configuration sett
 "pairlists": [
  {
  "method": "RemotePairList",
+ "mode": "whitelist",
+ "processing_mode": "filter",
  "pairlist_url": "https://example.com/pairlist",
  "number_assets": 10,
  "refresh_period": 1800,
@@ -194,14 +196,22 @@ The RemotePairList is defined in the pairlists section of the configuration sett
 ]
 ```
 
+The optional `mode` option specifies if the pairlist should be used as a `blacklist` or as a `whitelist`. The default value is "whitelist".
+
+The optional `processing_mode` option in the RemotePairList configuration determines how the retrieved pairlist is processed. It can have two values: "filter" or "append".
+
+In "filter" mode, the retrieved pairlist is used as a filter. Only the pairs present in both the original pairlist and the retrieved pairlist are included in the final pairlist. Other pairs are filtered out.
+
+In "append" mode, the retrieved pairlist is added to the original pairlist. All pairs from both lists are included in the final pairlist without any filtering.
+
 The `pairlist_url` option specifies the URL of the remote server where the pairlist is located, or the path to a local file (if file:/// is prepended). This allows the user to use either a remote server or a local file as the source for the pairlist.
 
 The user is responsible for providing a server or local file that returns a JSON object with the following structure:
 
 ```json
 {
  "pairs": ["XRP/USDT", "ETH/USDT", "LTC/USDT"],
- "refresh_period": 1800,
+ "refresh_period": 1800
 }
 ```
 

docs/includes/release_template.md

@@ -0,0 +1,37 @@
+## Highlighted changes
+
+- ...
+
+### How to update
+
+As always, you can update your bot using one of the following commands:
+
+#### docker-compose
+
+```bash
+docker-compose pull
+docker-compose up -d
+```
+
+#### Installation via setup script
+
+```
+# Deactivate venv and run 
+./setup.sh --update
+```
+
+#### Plain native installation
+
+```
+git pull
+pip install -U -r requirements.txt
+```
+
+<details>
+<summary>Expand full changelog</summary>
+
+```
+<Paste your changelog here>
+```
+
+</details>

docs/includes/showcase.md

@@ -0,0 +1,11 @@
+This section will highlight a few projects from members of the community.
+!!! Note
+ The projects below are for the most part not maintained by the freqtrade , therefore use your own caution before using them.
+
+- [Example freqtrade strategies](https://github.com/freqtrade/freqtrade-strategies/)
+- [FrequentHippo - Grafana dashboard with dry/live runs and backtests](http://frequenthippo.ddns.net:3000/) (by hippocritical).
+- [Online pairlist generator](https://remotepairlist.com/) (by Blood4rc).
+- [Freqtrade Backtesting Project](https://bt.robot.co.network/) (by Blood4rc).
+- [Freqtrade analysis notebook](https://github.com/froggleston/freqtrade_analysis_notebook) (by Froggleston).
+- [TUI for freqtrade](https://github.com/froggleston/freqtrade-frogtrade9000) (by Froggleston).
+- [Bot Academy](https://botacademy.ddns.net/) (by stash86) - Blog about crypto bot projects.

docs/index.md

@@ -63,6 +63,10 @@ Exchanges confirmed working by the community:
 - [X] [Bitvavo](https://bitvavo.com/)
 - [X] [Kucoin](https://www.kucoin.com/)
 
+## Community showcase
+
+--8<-- "includes/showcase.md"
+
 ## Requirements
 
 ### Hardware requirements

docs/requirements-docs.txt

@@ -1,6 +1,6 @@
 markdown==3.3.7
 mkdocs==1.4.3
-mkdocs-material==9.1.17
+mkdocs-material==9.1.19
 mdx_truly_sane_lists==1.3
-pymdown-extensions==10.0.1
+pymdown-extensions==10.1
 jinja2==3.1.2

docs/strategy-callbacks.md

@@ -750,7 +750,7 @@ class DigDeeperStrategy(IStrategy):
  # Hope you have a deep wallet!
  try:
  # This returns first order stake size
- stake_amount = filled_entries[0].cost
+ stake_amount = filled_entries[0].stake_amount
  # This then calculates current safety order size
  stake_amount = stake_amount * (1 + (count_of_entries * 0.25))
  return stake_amount

docs/telegram-usage.md

@@ -287,12 +287,17 @@ Return a summary of your profit/loss and performance.
 > **Best Performing:** `PAY/BTC: 50.23%`
 > **Trading volume:** `0.5 BTC`
 > **Profit factor:** `1.04`
+> **Win / Loss:** `102 / 36`
+> **Winrate:** `73.91%`
+> **Expectancy (Ratio):** `4.87 (1.66)`
 > **Max Drawdown:** `9.23% (0.01255 BTC)`
 
 The relative profit of `1.2%` is the average profit per trade.
 The relative profit of `15.2 Σ%` is be based on the starting capital - so in this case, the starting capital was `0.00485701 * 1.152 = 0.00738 BTC`.
 Starting capital is either taken from the `available_capital` setting, or calculated by using current wallet size - profits.
 Profit Factor is calculated as gross profits / gross losses - and should serve as an overall metric for the strategy.
+Expectancy corresponds to the average return per currency unit at risk, i.e. the winrate and the risk-reward ratio (the average gain of winning trades compared to the average loss of losing trades).
+Expectancy Ratio is expected profit or loss of a subsequent trade based on the performance of all past trades.
 Max drawdown corresponds to the backtesting metric `Absolute Drawdown (Account)` - calculated as `(Absolute Drawdown) / (DrawdownHigh + startingBalance)`.
 Bot started date will refer to the date the bot was first started. For older bots, this will default to the first trade's open date.
 

docs/trade-object.md

@@ -141,7 +141,8 @@ Most properties here can be None as they are dependant on the exchange response.
 `amount` | float | Amount in base currency
 `filled` | float | Filled amount (in base currency)
 `remaining` | float | Remaining amount
-`cost` | float | Cost of the order - usually average * filled
+`cost` | float | Cost of the order - usually average * filled (*Exchange dependant on futures, may contain the cost with or without leverage and may be in contracts.*)
+`stake_amount` | float | Stake amount used for this order. *Added in 2023.7.*
 `order_date` | datetime | Order creation date **use `order_date_utc` instead**
 `order_date_utc` | datetime | Order creation date (in UTC)
 `order_fill_date` | datetime | Order fill date **use `order_fill_utc` instead**