Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With the
prepare_*
functions we now have the possibility to multi-thread operator execution.I propose a minimal change in the
node_context
to make this possible and would like to discuss this:Instead of making the
executer_context
inside thenode_context
a list as discussed in #56 I would add a single pointer and flag to thenode_context
:By doing so we can introduce new nodes to the inference which are explicitly marked as threadsafe and may be executed in parallel to the current node.
The
prepare_*
functions can inject as many nodes as necessary and may even specify a different executer and context for each new node.When a node is encountered that is not threadsafe, the inference function needs to wait for all previous jobs to finish before executing the next: