ExchangeDelistFilter Feature

[jakubikan]

Describe the enhancement

Hey Guys I have written a delist filter for binance
https://binance-docs.github.io/apidocs/spot/en/#get-symbols-delist-schedule-for-spot-market_data

Not sure if it makes sense or if it actually will help but maybe you think about this and maybe this worthy to take into the mainline
It requires authentication so it might only run in not dry mode.

"""
Exchange Delist filter
"""
import logging
from typing import Any, Dict, List

import pandas as pd

from freqtrade.constants import Config
from freqtrade.exchange.types import Tickers
from freqtrade.persistence import Trade
from freqtrade.plugins.pairlist.IPairList import IPairList, PairlistParameter


logger = logging.getLogger(__name__)


class ExchangeDelistFilter(IPairList):

    def __init__(self, exchange, pairlistmanager,
                 config: Config, pairlistconfig: Dict[str, Any],
                 pairlist_pos: int) -> None:
        super().__init__(exchange, pairlistmanager, config, pairlistconfig, pairlist_pos)

        self._refresh_period = self._pairlistconfig.get('refresh_period', 1800)




    @property
    def needstickers(self) -> bool:
        """
        Boolean property defining if tickers are necessary.
        If no Pairlist requires tickers, an empty List is passed
        as tickers argument to filter_pairlist
        """
        return False

    def short_desc(self) -> str:
        """
        Short allowlist method description - used for startup-messages
        """
        return f"{self.name} - Removing Pairs when delisting."

    @staticmethod
    def description() -> str:
        return "Filter pairs by performance."

    @staticmethod
    def available_parameters() -> Dict[str, PairlistParameter]:
        return {
            **IPairList.refresh_period_parameter(),
        }

    def filter_pairlist(self, pairlist: List[str], tickers: Tickers) -> List[str]:
        """
        Filters and sorts pairlist and returns the allowlist again.
        Called on each bot iteration - please use internal caching if necessary
        :param pairlist: pairlist to filter or sort
        :param tickers: Tickers (from exchange.get_tickers). May be cached.
        :return: new allowlist
        """
        # Get the trading performance for pairs from database


        delistings = self._exchange._api.sapiGetSpotDelistSchedule()
        delisted_pairs = set(sum([d['symbols'] for d in delistings], []))

        for pair in pairlist:
            pair_in_delisting_format = pair.replace('/', "")
            if pair_in_delisting_format in delisted_pairs:
                pairlist.remove(pair)
                self.log_once(f"Removed {pair} from whitelist due to delisting", logger.info)


        return pairlist

[xmatthias]

in principle - absolutely not against this - as in - would be great to have it.
But i'd want some more abstraction (and as a Pull request 😆 ) - so it can be easily implemented for other exchanges (also, we'd need a "can only run on exchange X" blocker - currently it'll start on bybit, but will fail on bybit, as they don't have this endpoint (or rather, it's called differently).

That most likely means that it should be moved into the exchange class - with a flag that can "enable" this functionality from there (so adding new exchanges won't need a new filter).

you'll also want to implement some caching (and error-handling) i'd think (also for personal use) - filter_pairlist() is called once per iteration - so you're fetching the delist schedule roughly every 5s ... which will probably not cope well with your rate limits.

At least on binance, this seems to be pretty far into the future (current delistings happen on april 3rd) - so a caching of "once per hour" might suffice ...?

[stash86]

The issue with this approach is, for it to be effective, you need to refresh the list frequently. 30 mins is a long time. By the time your code fetch the new delist, it would have been down 30% for example. I'm not sure what is the best number, but 20-30 secs should be ideal. But that leads to the second issue.

Rate limit. If you only run 1 bot from 1 IP address, you have no issue. But once you run multiple bots, the weight of them pulling the info every 20-30secs gonna make your IP banned temporarily. We have unofficial binance proxy to fetch candle data through websocket, but it seems the delist announcement isn't available in websocket.

Personally, I would create a separate bot to check the delist, and send the list to all bots

[stash86]

As example, the most recent delist news was posted on 2024-03-20 06:01 on their telegram channel, and 2024-03-20 06:03 on their site. And the image below is how the market reacted. First long red candle is 06:01.

If you have a 5m tf mean reversion strat, your bot most likely would have bought the coin at 06:05, since your refresh period is 30mins (ofc depends on when the bot started). Is that what you want?

Latest 30m tf chart. The long red candle was a -46% dip at the time of the delist announcement

[jakubikan]

I have it currently running on 60sec on a 5mins timeframe could be enough to have this caught
Was just an idea to check with you first to get some addtional information.
I could handle the rest this evening if you want me to create a PR

[jakubikan]

Maybe we can implement this also on the dataprovider so we could actually use the call on the the confirm_trade_entry

and or custom_trade_exit

[xmatthias]

well that's the idea I'd think - yes, absolutely.

if we try to implement caching - I'd probably go with a way similar to how we use it in get_rate - where we push the data into an expiring cache - which people can configure the way they desire.

the _refresh_period in your case really does not work this way - as filter_pairlist is called anyway (handling should be done there - but i wouldn't go there - but move that to the exchange instead).