How to use grad-cam on audio classify

[ThinkInFuture]

Hi
from the project i get grad-cam is good for cv on Advanced AI explainability for PyTorch
i am wondering if this can be worked on audio classify , also i am trying on this
on audio part , also use CNN/ViT , usually first do MFCC on audio then send to Model
But in audio part , dont use cv2 for image read or write , instead is librosa
I not sure whether grad-cam can be used for audio classify explainability , hope any suggestion can given to me

[jacobgil]

Hi,
What does the model look like ?
What's the input and the output tensors in the model ?

[ThinkInFuture]

specgram can be show to image with code
print("Shape of spectrogram: {}".format(specgram.size()))
fig, ax = plt.subplots(figsize=(12, 3))
im = ax.imshow(specgram, aspect="auto", origin="lower",interpolation='none')
plt.colorbar(im, ax=ax)
plt.xlabel("Frames")
plt.ylabel("Channels")
plt.tight_layout()

[ThinkInFuture]

I want to use grad-cam to do an interpretable application on audio classification. For audio classification, I usually do not send audio to the classification network first, but extract the Mel spectrum feature, which is called fbank. (Spectrum, which can be drawn by librosa) I want to use grad-cam to label the features used in the classification network's spectrogram through heatmaps to help me analyze the network.

[ThinkInFuture]

I also wonder if other teams have tried to apply this to audio classification before.

[jacobgil]

Thanks.
How does the classification network look like?
Is it just a fully connected network?
What is its structure ?

[jacobgil]

does it use convolutions ?