In this prject we compare and evaluate two simple embedding models which can be constructed directly from a given co-occurrence matrix extracted from Twitter data; Positive Pointwise Mutual Information (PPMI), and Hellinger Principal Component Analysis (H-PCA). For each embedding model we consider three alternative metrics for word similarity: cosine, euclidean and manhattan distance.
Then, taking each combination of embedding model and similarity measure, we report results of two intrinsic evaluation measures, word similarity and concept categorization, on goldstandard datasets. We then qualitatively compare hierarchicalclustering dendrograms produced by the two most promising methods on sets of concept-categorized words, finding that the resulting dendrograms reproduce sensible semantic segmentations under both embedding types. click here for the full project report.
Of the metrics we tested, we consistently found that for each embedding model, the best performing was cosine similarity. Additionally we found that PPMI embeddings outperformed H-PCA embeddings under cosine similarity in both the word similarity and concept categorization quantitative tests.
And furthermore, in hierarchical clustering tests, we found that PPMI embeddings produced more semantically homogeneous clusters.
Although we found that the PPMI word embedding outperformed H-PCA in our results, the differences are marginal which is understandable given that both embeddings were drawn from the same numerical representation (a co-occurrence matrix).