Skip to content

dinhanhx/performance_calculation_tool_for_hm

Repository files navigation

Performance calculation tool for Hateful Memes Challenge

This simple project uses simple functions from pretty errors, click, pandas, sklearn. Therefore, one can go to these link and install as instructions. This works with Python 3.7

calc_test.py calculates AUC ROC and Accuracy scores. One can run python calc_test.py --help for instruction or can read the source code. It's very simple.

calc_test.py takes test_seen.jsonl (Phase 1) or test_unseen.jsonl (Phase 2) and result.csv. Importantly, test_seen.jsonl, test_unseen.jsonl must have labels.

result.csv must have to three columns:

  • Meme identification number, id
  • Probability that the meme is hateful, proba (must be a float)
  • Binary label that the meme is hateful (1) or non-hateful (0), label (must be an int)

Other scripts

combine.py is meant to combine all train.jsonl, dev_seen.jsonl, dev_unseen.jsonl, test_seen.jsonl, test_unseen.jsonl into data_test.jsonl. Importantly, test_seen.jsonl, test_unseen.jsonl must have labels. By combining so, data_test.jsonl contains all metadata of all memes in the dataset.

Releases

No releases published

Packages

No packages published

Languages