Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include the strategy.py and config.json used for each backtesting result. #8448

Open
Guscccc opened this issue Apr 3, 2023 · 3 comments
Open
Labels
Backtest Issues related backtesting. Enhancement Enhancements to the bot. Get lower priority than bugs by default.

Comments

@Guscccc
Copy link

Guscccc commented Apr 3, 2023

Describe the enhancement

I think it would be nice to include the strategy and config that's used for each backtest in the backtesting archive file for reproducibility. or to offer a command line option that does it when using freqtrade backtesting.

@xmatthias xmatthias added the Backtest Issues related backtesting. label Apr 3, 2023
@xmatthias
Copy link
Member

xmatthias commented Apr 3, 2023

Please get used to filling out the issue template (also if you THINK it's not relevant for your case). Those who frequently omit the template may have their issues closed without comment.
We're asking some very basic questions - by deleting the issue template, you're explicitly choosing not to answer our questions.

Filling out the template helps reduce our workload, and we greatly appreciate that.


i'm not entirely sure i see the point of this if i'm honest.

It'll bloat backtest results for sure - but what for exactly?
it's quite common to run 100ds of backtests with small, iterative changes
so we'd do a copy of the strategy, strategy parameters, config - and maybe the whole strategy directory (who knows what was imported from "next door") - but with what end-goal exactly (what are we trying to accomplish by bloating results)?

to have FULL reproducibility, we'd need to save the data directory as well (i could be missing some data at the end on one pair (for whatever reason) - and then download the missing parts - which would change results, too) - but then it really becomes ridiculous, really

@xmatthias xmatthias added the Enhancement Enhancements to the bot. Get lower priority than bugs by default. label Nov 30, 2023
@mexxBf
Copy link

mexxBf commented Jan 24, 2024

I believe implementing this approach would be highly beneficial. Otherwise, reproducing previous outcomes becomes a challenging task. In instances where I modify my strategies and later struggle to recall the specific changes, it hinders my ability to replicate past results. This is particularly true in scenarios involving hundreds of alterations. An even more effective solution would be to create a comprehensive overview, presenting the results in a list. A good place would be the list of the past backtests.

@stash86
Copy link
Contributor

stash86 commented Jan 24, 2024

Or you can just practice a good way of archiving files. Keep copies of old versions after several updates. Or use different class names after several updates. There are plenty of ways to do that.

As noted above, saving files used in backtest gonna eats your hard disk space faster than you can imagine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backtest Issues related backtesting. Enhancement Enhancements to the bot. Get lower priority than bugs by default.
Projects
None yet
Development

No branches or pull requests

4 participants