Skip to content

ict-bigdatalab/FER

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

FER

This is the source code of EMNLP'23 paper "From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification".

Retrieval-enhanced methods have become a primary approach in fact verification (FV); it requires reasoning over multiple retrieved pieces of evidence to verify the integrity of a claim. To retrieve evidence, existing work often employs off-the-shelf retrieval models whose design is based on the probability ranking principle. We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence. We introduce the feedback-based evidence retriever(FER) that optimizes the evidence retrieval process by incorporating feedback from the claim verifier. As a feedback signal we use the divergence in utility between how effectively the verifier utilizes the retrieved evidence and the ground-truth evidence to produce the final claim label. Empirical studies demonstrate the superiority of FER over prevailing baselines.

Table of Contents

Setup

Clone as follows:

git clone https://github.com/hengran/FER.git
cd FER
pip install -r requirements.txt

Download model parameter and data from [model_path] to to folders save_model/ and data/

Usage

Train FER models

python FER.py

Reproduce our results

  1. Download the trained model.
  2. Run the code
python evaluated.py

Citation

Please cite our paper if you use this code in your work:

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages