All pkl files in vsitu_vocab directory

[cws7777]

Hi :)

Where can I find all pkl files (verb_id_vocab.pkl and bpe_with_seps_vb_arg_vocab.pkl) in vsitu_vocab directory?
Should I download those? (I couldn't find a download link or instruction from github)

Thanks! :)

P.S.
I just tried to run with the command (CUDA_VISIBLE_DEVICES=#GPUs python main_dist.py "test") but gave me an error "the file doesn't exist".
I checked the vsitu_cfg.yml file and there are two file paths in ds.vocab_files

[TheShadow29]

Whoops, I forgot to add the link. Here it is: https://drive.google.com/file/d/1TAreioObLGKqU7M9wmnuaXh4b5s_2YdK/view?usp=sharing

They should be placed under data/vsitu_vocab

[cws7777]

Thanks! :)
Also, can you give me an example command of reproducing results (e.g. for any verb prediction model), if you don't mind?
It might be really helpful to follow up!

Best,
Paul

[TheShadow29]

Sure. In hindsight I should have added the commands (will do so sometime today).

In most cases, the command can be found under the log file under cmd with the associated experiment (usually searchable via ctrl+f)

Here is one for I3D+NLN (to train from kinetics pre-trained:

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main_dist.py "vbpred_i3d_nln" --train.bs=8 --train.bsv=8 \
--task_type="vb" --mdl.mdl_name="sf_base" --mdl.sf_mdl_name="i3d_r50_nl_8x8" --debug_mode=False \
--train.save_mdl_epochs=True --train.resume=False --mdl.load_sf_pretrained=True

Note that you must have the relevant kinetics pre-trained file under weights/ The exact path can be found in the relevant configs/vsitu_mdl_cfgs/ These are taken from SlowFast repository.

To only do the validation using a trained model (for instance using the model https://drive.google.com/file/d/1VqkFgCTSbl7aTsHFbtI9iQi_ds-nRbdN/view?usp=sharing)

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main_dist.py "vbpred_i3d_nln_only_valid" --train.bs=8 --train.bsv=8 \
--task_type="vb" --mdl.mdl_name="sf_base" --mdl.sf_mdl_name="i3d_r50_nl_8x8" --debug_mode=False \
--train.resume=True --train.resume_path=$MDL_PATH --only_valid=True

[cws7777]

Oh.. I see!
I should look at the log file more carefully...

Thank you for the instruction! :)

[cws7777]

@TheShadow29 Sorry for bothering! :'(

I'm getting error when I run the command that you gave me above (training from kinetics pre-trained).
The error(ModuleNotFoundError: No module named 'transformers.tokenization_gpt2') occurs in loading the pickle file of vocab which is bpe_with_seps_vb_arg_vocab.pkl. Is there something I missed on installing packages?

I checked with the console that transformers package is installed and worked properly. However, it gives an error when I load the file like below pic.

Thanks! :)

[TheShadow29]

@cws7777 This is strange. Can you confirm transfomers library is installed? You can use the command conda list |grep transformers. It should output: transformers 3.3.1 pypi_0 pypi

[cws7777]

@TheShadow29 My version of transformers (the output of conda list |grep transformers) is 4.5.1. Should this be a possible error? --> Wait, let me check with the version 3.3.1 :)
Updated: Oh, the version of transformers was the problem.

There are minor errors, but solved.

1. line 141 in dat_loader.py (self.cfg.video_frms_tdir --> should be self.cfg.video_frm_tdir)
2. In vsitu_cfg.yml, the value of video_frm_tdir should be changed. (from './data/vsitu_frames/' to ./data/vsitu_video_frames_dir)

Now I have to get the relevant kinetics pre-trained file from SlowFast..!

Thanks!

[TheShadow29]

Ah my bad. I will fix the issue. Thanks for the note.

Slowfast files can be downloaded from here: https://github.com/facebookresearch/SlowFast/blob/master/MODEL_ZOO.md

In particular, you should use Slowfast R50 8x8 and I3D-NLN R50 8x8.

Let me know if any hiccups arise!

[cws7777]

Thanks for the instruction! :) I already downloaded from the link that you mentioned! :)

The file name was a bit different though (in .yml fille, it has _CU_. But not in SlowFast).