Apply GradCam to Cnn+LSTM

[gabarlacchi]

Hi all,
I am working with a feature extractor (Inception V3, VGG16, whatever) plus an LSTM for sequences classification (let's say 3 sec. each one). There's any way to use GradCam in order to obtain areas activation on the input sequences? Right now, I just classify one sequence without single frame classification, is still possible apply GradCam to this network?
I didn't find any example code about it, but just for GradCam applied to single feature extractor.
Thanks in advance for the help,
Gabriele

[monjurulkarim]

@barloccia Did you find any solution?

[gabarlacchi]

Hi,
I used the standard way by applying grad-cam to each frame within the sequence. Since the CNN+LSTM network internally predict the class for each frame, you can then apply grad-cam to them. In my case this represents the area within every image in which the network focused at prediction time.
I applied the implementation of grad-cam tensorflow 2: https://github.com/ismailuddin/gradcam-tensorflow-2

Hope this helps,
Best