Vector-Semantics-Evaluation

This repository contains code for evaluating the alignment between lexical semantics and vector semantics using the SimLex-999 dataset and the Brown corpus from NLTK. The evaluation is conducted using various vector semantic methods, including TF-iDF and Word2Vec, and the results are compared against manually determined lexical semantics.

Introduction

Lexical semantics and vector semantics are two approaches to understanding the meaning of words in natural language processing. This project aims to evaluate how well vector semantic methods capture the semantics of words compared to manually determined lexical semantics.

Datasets

• SimLex-999: Golden standard dataset for lexical semantics evaluation.
• Brown corpus: Large corpus from NLTK used for training vector semantic methods.

Usage

To run the evaluation code:

1. Just download the SimLex-999 file and give path in the ipynb file.
2. Run the provided Python ipynb file, specifying the desired settings for vector semantic methods for both TF-IDF and Word2Vec.