Skip to content

A simple version of DeepRank & MatchPyramid implemented in PyTorch.

License

Notifications You must be signed in to change notification settings

pl8787/DeepRank_PyTorch

Repository files navigation

DeepRank_PyTorch

A simple version of DeepRank & MatchPyramid implement in PyTorch.

Please reference paper: DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval [https://arxiv.org/abs/1710.05649]. Text Matching as Image Recognition [https://arxiv.org/abs/1602.06359].

Quick Start

Toy Dataset Download

You can download data from [here].

$ tar xzvf data.tar.gz

Dataset Format

Word Dictionary File

(eg. word_dict.txt)

We map each word to a uniqe number, called wid, and save this mapping in the word dictionary file.

For example,

word   wid
machine 1232
learning 1156

Corpus File

(eg. qid_query.txt and docid_doc.txt)

We use a value of string identifier (qid/docid) to represent a sentence, such as a query or a document. The second number denotes the length of the sentence. The following numbers are the wids of the sentence.

For example,

docid  sentence_length  sentence_wid_sequence
GX000-00-0000000 42 2744 1043 377 2744 1043 377 187 117961 ...

Relation File

(eg. relation.train.fold1.txt, relation.test.fold1.txt ...)

The relation files are used to store the relation between two sentences, such as the relevance relation between query and document.

For example,

relevance   qid   docid
1 3571 GX245-00-1220850
0 3571 GX004-51-0504917
0 3571 GX006-36-4612449

Embedding File

(eg. embed_wiki-pdc_d50_norm)

We store the word embedding into the embedding file.

For example,

wid   embedding
13275 -0.050766 0.081548 -0.031107 0.131772 0.172194 ... 0.165506 0.002235

Training & Evaluation

Run the following commond to train & evaluate DeepRank and MatchPyramid model on Letor dataset:

$ jupyter notebook
$ # open sandbox-*.ipynb

Requirements

  • Python 3.6
  • TensorFlow 1.1.0

About

A simple version of DeepRank & MatchPyramid implemented in PyTorch.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published