Why not torchvision.io.read_video instead of cv2.Videocapture()

[Rezowan-Ferdous]

Thank you for your great work,

I am extracting I3D features from a dataset where frames and labels are annotated in a separate text (start_frame end_frame label). I was just wondering if I could change the cv2.videocapture with read_video of torchvision to extract from the subparts of a video. Will there be any difference?

[v-iashin]

good question!

it should be fine but I haven't rigorously tested it. please try it and let me know. to make sure it works, specify the show_pred for a video or two to see if predictions make sense at all and, ideally, match the ones obtained with cv2 extractor.

the reason why it is implemented this way is because of the legacy code. i know it works, because I used it for a few projects, and I didn't want to break it.

At the same time, the discrepancy in io across feature extractors disappoints me. maybe I will unify it at some point.

there is an advantage of using cv2 over read_video and it is memory efficiency as cv2 discards stacks of frames after extracting one feature vector while with read_video it loads the whole video in ram and performs feature extraction. this means if you have an hour-long video, the extraction will fail

[Rezowan-Ferdous]

that make sense, thanks