Training error in VIDSTG.

[liujiaheng]

Have you met this error?

[antoyang]

I haven't. You may verify that you use the suggested FFMPEG version, that you set up the path to the videos correctly in the config files and that the videos are properly downloaded.

[antoyang]

The folder "vidstg_vid_path" (in the config files) should contain a folder "video" in which there are the different folders containing VidSTG videos ("0000" etc).

[liujiaheng]

I have run for 500 iters successfully. This error happens randomly.

[antoyang]

As I haven't met this error, I would suggest looking at which videos are problematic and what's the difference between len(images_list) and len(frame_ids) for these videos.

[liujiaheng]

Could you provide the download link of the ffmpeg (4.2.2 amd64 static)?
Besides, I also suppose the generated annotation json files after processing may be not accurate. Could you provide the generated json file?

Thanks.

[PeiqinZhuang]

As I haven't met this error, I would suggest looking at which videos are problematic and what's the difference between len(images_list) and len(frame_ids) for these videos.

Hi, I also encountered some problems, which are the same as those of liujiang. Besides, the error also occurred in this place .

TubeDETR/engine.py

Line 107 in 2f15abf

assert len(targets) == len(outputs["pred_boxes"]), (

Since the authors added assertion here, I am not sure if you have filtered some failure videos. If possible, would you mind sharing your clean data list?

[TreezzZ]

Same problem.

As I haven't met this error, I would suggest looking at which videos are problematic and what's the difference between len(images_list) and len(frame_ids) for these videos.

Hi, I also encountered some problems, which are the same as those of liujiang. Besides, the error also occurred in this place .

TubeDETR/engine.py

Line 107 in 2f15abf

assert len(targets) == len(outputs["pred_boxes"]), (

Since the authors added assertion here, I am not sure if you have filtered some failure videos. If possible, would you mind sharing your clean data list?

Same problem. Have u solved it?

Thx.

[johnbager]

收到

[Awj2021]

Have you met this error?

Hi, I met the same error, did you fix this problem? Thanks.

[salmank255]

I printed the lengths of frame_ids and image_list and got this:

AssertionError: len(images_list) 131
len(frame_ids) 130
for /video/1101/6797913724.mp4

This means one extra frame has been created by ffmpeg.

I fixed it by adding the below (conditions removing the extra frame) after this line

TubeDETR/datasets/vidstg.py

Line 115 in 2f15abf

images_list = np.frombuffer(out, np.uint8).reshape([-1, h, w, 3])

    if len(images_list) > len(frame_ids):
        images_list = images_list[:len(frame_ids)]
    if len(frame_ids) > len(images_list):
        frame_ids = frame_ids[:len(images_list)]

[antoyang]

@liujiaheng @PeiqinZhuang I do not remember if this is the link I used for downloading it at the time but https://www.johnvansickle.com/ffmpeg/old-releases/ seems to provide the FFMPEG version I used for this project. I did not filter any video after the preprocessing script, the assertion was mainly for some initial checks. @salmank255 Did you use the FFMPEG version mentioned in the readme? In all problematic cases was it always one more frame created by FFMPEG?

[salmank255]

No, I used ffmpeg-python and the error of frames mismatch seems to have appeared only once after 200 iterations. Now, the next problem is the same faced by @PeiqinZhuang and @TreezzZ after 2700 iterations we are getting the same assertion error where the targets len is equal to 0.

[antoyang]

For the targets len 0, I am wondering if the videos are not corrupted. Can you ffprobe them to check? Otherwise a turnaround is to use FFMPEG to extract all frames of videos (5fps) once for all and to load each of them with PIL in the dataloader. It would make it easy to check if there is an issue with any video. I did this in an early version of the project and it was giving similar results.

[PeiqinZhuang]

No, I used ffmpeg-python and the error of frames mismatch seems to have appeared only once after 200 iterations. Now, the next problem is the same faced by @PeiqinZhuang and @TreezzZ after 2700 iterations we are getting the same assertion error where the targets len is equal to 0.

Hi, you can just filter out those fault videos in advance. Unfortunately, no matter what version of ffmpeg I use, there are always some fault videos. You can use those codes in datasets, i.e. how to decode each video, to check every video before you run the code.

[salmank255]

Hi @PeiqinZhuang

The faulty videos are 7003314632 and 9731127827. I removed both of them from the train annotation file and run the code now it's working fine.

[salmank255]

You also need to include below two conditions otherwise there will be an assertion error for frames lens

if len(images_list) > len(frame_ids):
    images_list = images_list[:len(frame_ids)]
if len(frame_ids) > len(images_list):
    frame_ids = frame_ids[:len(images_list)]