-
Notifications
You must be signed in to change notification settings - Fork 1
Determine the degree of relevance between two input questions by utilizing Bidirectional Encoder Representations from Transformers (BERT) provided in Hugging Face & Deep Neural Network model built in Pytorch
pj1114/Relevant-Question-Detection
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
This project is group project for COMPSCI585 - Natural Language Processing ########## Group Members ########## - Ye Zhang - YuanJar Wang - KaiYuan Deng - YuXin Huang ########## Code Running ########## All our code is written in google colab notebook, the link of it is: https://colab.research.google.com/drive/1fauoQi2MOm7Q3_VRglWdfxXeWbCFBIjA#scrollTo=7jZTDU3nshLe To reproduce our experiments, just run the notebook from start to end. However, there's a section which connects to Ye's personal google drive. This section is used to save models and training history, also load them. Therefore, if you want to load our final model, you need to change the code in load checkpoint section. The link of our final model is: https://drive.google.com/file/d/1-UAtlUtuYxiwM0UQ2rprDWrnlsksaWu5/view?usp=sharing You can load it by the same way as how you load the data in the load data section. ########## Datasets ########## All datasets used in our project have been uploaded to google drive and can be accessed by these shared links: * Quora question pairs: - quora.pkl: https://drive.google.com/file/d/1lm8eXR3ME2VYwSlB-2XmeuKH_6EVoj4j/view?usp=sharing - train_data.pkl: https://drive.google.com/file/d/1-lil_uwz1aXsHBYKtDDsn2wWkCxQPFRH/view?usp=sharing - valid_data.pkl: https://drive.google.com/file/d/1-zCou-9eOBC4n9ebEKcsQdRTjy2U4cPG/view?usp=sharing - test_data.pkl: https://drive.google.com/file/d/1-oKdsv4Moxbs8CP3ngmAg_-zYjlKN52S/view?usp=sharing * StackExchange question pairs: - BodyText.pkl: https://drive.google.com/file/d/1zV5pKCoQ2uyIoS5v_IRFXq6Wl6F-W7Dr/view?usp=sharing - TopicText.pkl: https://drive.google.com/file/d/144lRBGpzjrdEJo26tfaAwOYFwW9QNfHL/view?usp=sharing Note that all these links are attached into our notebook, you can directly run the notebook to download them.
About
Determine the degree of relevance between two input questions by utilizing Bidirectional Encoder Representations from Transformers (BERT) provided in Hugging Face & Deep Neural Network model built in Pytorch
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published