Possibility to reverse marginal positions (long/short) at a single candle

[a18]

Describe your environment

(if applicable)

• Operating system: Windows10
• Python Version: 3.10
• CCXT version: 4.0.89
• Freqtrade Version: 2023.6

Describe the enhancement

Currently, Freqtrade does not seem to support a rather convenient feature: the ability to reverse a marginal position (e.g., futures) from long to short (or vice versa) within the same candle. For example, in the Binance GUI, this functionality is implemented in a single click of "Reverse" button.

The absence of this feature can be quite disappointing, as it may lead to unnecessary delays and, in some cases, financial losses when a strategy needs to wait for the next candle to open the second half of a reverse transaction. This is particularly impactful on longer timeframes, such as 1h and longer.
As a workaround some people even suggest to launch two FreqTrade instances simalteneously(!)

Additionally, this limitation can lead to confusion during backtesting, as Freqtrade, without issuing any warning in the log, will ignore "Enter long" signal that is placed in the same candle with "Exit short" signal, despite the documentation suggesting that such cases are not forbidden:

The following rules apply, and entry signals will be ignored if more than one of the 3 signals is set:

• enter_long -> exit_long, enter_short
• enter_short -> exit_short, enter_long

(according to this, enter_long should not conflict with exit_short)

Anyway, thanks for the great product!!

[xmatthias]

in live, it's an artificial limitation (to some degree) - which could be fixed with 1-2 lines.

In backtesting, this is however built into the execution logic, which automatically locks pairs for the duration of the candle (since we check for entries before exits).
Reversing this logic will introduce other limitations (you could no longer exit on the entry candle).

This leaves the possibility to check for "entries -> exits -> entries" - but then the 2nd entry will no longer be able to exit within this candle - so the 2nd entry behaves different to the initial entry (so we'd need to check "entries -> exits -> entries -> exits" - but even here it's limited, as you could have more than 2 entries per candle i guess).

This would introduce semi-infinite loops to backtesting - with a performance impact that in my opinion doesn't justify the usefulnes.

If you have some ideas on HOW this can be implemented in backtesting - share your ideas and i'm happy to look at it (ideally to review a PR for it).

Otherwise, i don't see this as something we'll look at anytime soon (most likely never).

[a18]

Thanks for the very informative comment! I feel, this question is being discussed for a long time.

Could you, please, elaborate the phrase "you could have more than 2 entries per candle i guess"?
I thought that in a strategy we just append to the "dataframe" these 4 columns: "enter_long", "enter_short", "exit_long", "exit_short", that could have only 0 and 1 values, resulting in 16 combinations, with most of them treated as conflicting according to the current quite reasonable documentation rules.

[xmatthias]

That's for static / dataframe based signals - yes - though the 16 combinations are not really relevant, as most would conflict.

but for all other (usually more common) exit methods (stoploss, roi, custom_exit, ...) - there's no such thing as collisions, they're always enacted (There's exceptions to this, irrelevant for this issue).
so while you can't have the case of "entry, exit, enter short" on the same candle (conflicting signals, the initial entry would be ignored) - it still doesn't change the problematic.

• we must check for entries first, followed by exits (this ensures we are able to exit within that candle on a non-dataframe based exit).
• To facilitate an entry AFTER the exit within the same candle, we must check for entries again - followed by checking for exits again.

Assuming 1h candles, we're on the 11h candle (stoploss is taken as sample, all above mentioned non-DF exit methods are treated equally).

The min. scenarios you'd aim to cover are

• Trade opened at 11, stays open
• Trade opened at 11, exits on stop within that same candle
• trade opened at 9, exits on stoploss at 11, entry in other direction in same candle (stays open)
• trade opened at 9, exits on stoploss at 11, entry in other direction in same candle (stays open), another stop on same candle
• trade opened at 9, exits on stoploss at 11, entry in other direction in same candle - but is ignored because another pair was ahead in the pairlist and filled this trade slot (just because you have a close with entry signal doesn't mean this entry is the next one to look at)

Scenarios to ignore due to conflicts (that's the easy part)

• Trade opened at 11, exits on stop within that candle, entry in other direction in same candle

That's at minimum 2 iterations you'd have for each candle, each containing

• loop through ALL pairs (respecting pairlist sequence)
• full entry logic
• full exit logic
• all order handling, in cases where orders don't immediately fill
• splits into detail timeframe eventually (not sure how this needs to be handled)

You could also have 5 trades (on different pairs) exiting at the same candle, with 5 new trades entering, 3 on the same pair reversing position, 2 on new pairs - and one of the "reverse position" signals being ignored, as the new pairs are further in front in the pairlist.

All of the above logic currently assumes that it's executed exactly once per candle - where executing it twice will have

• severe, negative implications on performance
• avoid calling the same callback twice for no reason (not just for performance reasons, but also because people tend to have side-effects in their callbacks)
• ...

As this will only benefit a rather small percentage of users, all of this needs to be implemented in a way that's not impacting performance negatively for users that don't need this.
Good luck :)

[a18]

Aha, as I supposed, I just have not thought about "non-DF" exits. You are right, the things seem rather complex.

Currently I don't have enough expertise to suggest working concept/PR, but here's a table of these 16 combinations with corresponding actions, that could seem reasonable (maybe, for future implementations/fixes). Please, feel free to edit/copy it, etc.
Terminology:
SC (Soft collision) - the combination is strange, but do NOT result in missing a trade.
HC (Hard collision) - the combination is strange, and could result in missing of a trade.
CurPos - current position

id	enter_long	enter_short	exit_long	exit_short	Action for the next candle
1	0	0	0	0	None
2	0	0	0	1	CurPos is None -> Log SC CurPos is Long -> Log HC CurPos is Short -> Close short
3	0	0	1	0	CurPos is None -> Log SC CurPos is Long -> Exit long CurPos is Short -> Log HC
4	0	0	1	1	CurPos is None -> Log SC CurPos is Long -> Exit long and log SC CurPos is Short -> Exit short and log SC
5	0	1	0	0	CurPos is None -> Enter short CurPos is Long -> Log HC CurPos is Short -> Log SC
6	0	1	0	1	Log HC
7	0	1	1	0	CurPos is None -> Enter short + log SC CurPos is Long -> Exit long + enter short CurPos is Short -> Log SC
8	0	1	1	1	Log HC
9	1	0	0	0	CurPos is None -> Enter long CurPos is Long -> Log SC CurPos is Short -> Log HC
10	1	0	0	1	CurPos is None -> Enter long + log SC CurPos is Long -> Log SC CurPos is Short -> Exit short + enter long
11	1	0	1	0	Log HC
12	1	0	1	1	Log HC
13	1	1	0	0	Log HC
14	1	1	0	1	Log HC
15	1	1	1	0	Log HC
16	1	1	0	1	Log HC

EDIT: as there are "non-DF" exits, the word "strange" is not too good, as the case could be quite normal.

[xmatthias]

Unfortunately that table is mostly useless (at least for this issue implementation) - as it ignores non-dataframe behavior (which is where most of the complexity is introduced, as you can no longer predict where a trade is going to exit).
It's essentially just "exploding" the collision rules.

as long as we stick to "dataframe only" behaviors - behavior is clear - but we can't.

One thing to note:
Backtesting will not write logs for random actions within the simulation. Rendering Terminal output is rather slow - so adding "logging" here would be quite bad for performance.

[a18]

Yes, logging of SC (soft-collisions) could be reasonable only on -v (or -vv) versbosity levels or just skipped. But logging of HC (hard collisions) as warnings - in my opinion, is vital as they report principal strategy inconsistencies to the user.

[a18]

Here's the version 2 of the table - now including "non-DF" exits.

Terminology:
SC (Soft collision) - the combination is strange, but do NOT result in missing a trade.
HC (Hard collision) - the combination is strange, and could result in missing of a trade.
CurPos - current position ("Exited" means that the position was closed by stop loss, etc.)

id	enter_long	enter_short	exit_long	exit_short	Action for the next candle
1	0	0	0	0	None
2	0	0	0	1	CurPos is Exited -> ok CurPos is None -> Log SC CurPos is Long -> Log HC CurPos is Short -> Close short
3	0	0	1	0	CurPos is Exited -> ok CurPos is None -> Log SC CurPos is Long -> Exit long CurPos is Short -> Log HC
4	0	0	1	1	CurPos is Exited or None -> Log SC CurPos is Long -> Exit long and log SC CurPos is Short -> Exit short and log SC
5	0	1	0	0	CurPos is Exited or None -> Enter short CurPos is Long -> Log HC CurPos is Short -> Log SC
6	0	1	0	1	Log HC
7	0	1	1	0	CurPos is Exited -> Enter short CurPos is None -> Enter short + log SC CurPos is Long -> Exit long + enter short CurPos is Short -> Log SC
8	0	1	1	1	Log HC
9	1	0	0	0	CurPos is Exited or None -> Enter long CurPos is Long -> Log SC CurPos is Short -> Log HC
10	1	0	0	1	CurPos is Exited ->Enter long CurPos is None -> Enter long + log SC CurPos is Long -> Log SC CurPos is Short -> Exit short + enter long
11	1	0	1	0	Log HC
12	1	0	1	1	Log HC
13	1	1	0	0	Log HC
14	1	1	0	1	Log HC
15	1	1	1	0	Log HC
16	1	1	0	1	Log HC

Hope this could be useful for further discussions sometimes in the future.

[maddrid]

my 2 cents here : the ability to reverse a marginal position (e.g., futures) from long to short (or vice versa) within the same candle should be handled from config first ..
reverse_position = false ( default for 99 % use cases) ..
but in some cases , let the user decide about reverse_position and implement his "strategy reversion " dependent on the exchange ( some require 2 orders ( close the current position, and second order to take the opposite side ) other manage the reversion on the exchange level
reverse_position = true
no more hacking the core and nasty inventions to do a simple reversion

[MF1B]

Wouldn't it be a simple case of processing the exities first and then the entries?

[stash86]

Dry/live, yes it's simple.

Backtest? Read the replies above. Ofc if you think you can make it work, please do open a PR

[MF1B]

I'm sorry! I didn't know that backtesting was more important than dry/live. Could we consider the open candle, after the signal, as the closing/opening value, for backtest purposes only, not being as precise as necessary?

[stash86]

once again, read above replies. Matthias already explained the issues. If you think you can provide the solution, please do open a PR.

If you only care about dry/live, you can manually unlock the pair. You can read the docs on how to do that