RoI head that considers pairs of proposals?

[danielbulhosa]

I am interested in using or developing an RoI head that takes a series of bbox proposals and predicted classes, takes their Cartesian product, extracts features for each bbox in the pair, and passes those along with the predicted classes to a model that makes some prediction about the pair of regions (e.g. is there a relation between the two regions?).

Is there an existing RoI head that does something like this? Looking at the base interface for BaseROIHead it appears its child classes' methods (in particular forward_train):

1. Do not ingest class predictions from previous layers.
2. Do not have support for pairs of bounding boxes.

If there is not an existing RoI head for this, what would be the best way to implement it? Should one extend an existing ROIHead class or create a separate family of classes with a different interface (i.e. not inheriting from BaseROIHead)? I assume the latter would be better rather than trying to force this into the existing interface. If so, what would these classes have to output and what methods do they need to implement to work with the rest of the framework? Or would the model building, training, and inference scripts need to be updated to make this work? Thanks for your insight!