Return df based on features used in strategy creation

[SW4T400]

Hi,

I am wondering what your oppinions are on the following and what your workflows are:

Currently,

the only output (and only when using a rule) is the TestResult object:

strategy.add_execution(buy_low, ['PG', 'AMD'])
result_with_trades_etc = strategy.backtest()
And when we dont use a rule we get None as an output:

strategy.add_execution(None, ['NVDA'], models=model_slr0)
this_is_None = strategy.walkforward(warmup=20,windows=3,train_size=0.5,lookahead=1,calc_bootstrap=False)

In my workflow

I would like for the strategy object (when calling backtest or walkforward) to return data based on the provided inputs. So wether I use a rule, indicator, model or ANY combination of the above (or even just one of them) I would like to return relevant data.

After asking for it here I have (very crudely) implemented this in my fork (see this example file) for illustration purposes:

with a rule	strategy.add_execution(hold_long0, ['NVDA'], models=model_slr0)	df, result = strategy.walkforward(..)
without a rule	strategy.add_execution(None, ['NVDA'], models=model_slr0)	df = strategy.walkforward(..)

So depending on which elements are used in the strategy I get a df containing the relevant infos:

Features used in strategy creation	Columns in output dataframe
rule	bardata
rule + indicator	bardata, indicator values
rule + model	bardata, predictions
rule + indicator + model	bardata, indicator values, predictions
indicator	bardata, indicator values
model	bardata, predictions
indicator + model	bardata , indicator values, predictions

Implementation

Is this something you guys find useful or are you using pybroker differently?
Incase this is interesting maybe someone could add this feature? As you can see from my fork, my programming skills are not even close to @edtechre and I don´t understand the sourcecode well enough to be able to do this at the moment.

Visualizations

In any case I will be using my crude implementation to start developing visualizations (based on dash/plotly) for the rule, indicator & model constellations shown above (indicator-, model-, trading rules performance-, portfolio view etc.).

Has anyone else developed any such visualizations and would like to share?

Kind regards,
Stefan

[edtechre]

Thank you Stefan. I like this suggestion of returning indicator values and model predictions when they are used. I will open an issue to work on this.

[edtechre]

Filed #84

[edtechre]

Hi @SW4T400,

Looking at this more, it is not feasible to have a single DataFrame that stores bardata, indicators and predictions because these would apply to multiple symbols. However, it could make sense to return a dictionary of DataFrames keyed by symbol. This dictionary would be a member of TestResult, and TestResult would always be returned when calling backtest or walkforward.

[edtechre]

The change to support this has been committed to the dev branch. I added a new return_signals config option. When enabled, TestResult#signals will hold bar data, indicator data, and model predictions.

The change is targeted for v1.1.31 release.

[SW4T400]

Wow, thank you so much!

[SW4T400]

Would it make sense to also see the train predictions in order to inspect how the "training" went?
I have to admit I haven´t even started developing ideas, but I believe this might be useful in some workflows.

[edtechre]

PyBroker never runs inference on the model using the training data. I believe this would best be handled in your own train function as part of your training evaluation.