| LLMs4OL Paradigm | Task A: Term Typing | Task B: Type Taxonomy Discovery | Task C: Type Non-Taxonomic Relation Extraction | Finetuning | Task A Detailed Results | Task B Detailed Results | Task C Detailed Results | Task A Datasets | Task B Datasets | Task C Datasets | Finetuning Datasets |
- Task Definition: For a given term types, identify and extract “non-is-a” or semantic relationships between types.
- Task Goal: To discover non-taxonomic semantic heterarchical relations between types.
- Evaluation Metric: F1-Score
To run zero-shot testing you can try the following command line after you are done with installing requirements:
ptyhon3 test.py [-h] --kb_name KB_NAME --model MODEL --device DEVICEWhere KB_NAME, MODEL, TEMPLATE, and DEVICE accept the following values:
KB_NAME:
umls
MODEL:
bert_large, flan_t5_large, flan_t5_xl, bart_large, gpt3, bloom_1b7, bloom_3b, llama_7b, chatgpt, gpt4
DEVICE:
cpu, cuda
As an example run if you want to run your model on the umls dataset with the bert_large model and I have GPU resource, the command line would be:
python3 test.py --kb_name="umls" --model="bert_large" --device="cuda"Or you can easily run the test_auto.sh script to run models on umls dataset:
./test_auto.sh