-
Notifications
You must be signed in to change notification settings - Fork 8
Project Information and Overview
Grading essays is a time-consuming task for university professors, often leading to delays in providing feedback to students. Additionally, subjective biases may influence grading consistency. To address these challenges, we aim to develop an automated essay grading system powered by Natural Language Processing (NLP) techniques. This system will enable professors to assess students' essays efficiently, provide timely feedback, and identify areas for improvement, ultimately enhancing the grading process and learning outcomes.
Key features include user authentication to ensure security and privacy, essay submission supporting text or document formats, and rigorous analysis including word count verification, keyword identification, and fact-checking capabilities. Additionally, the system evaluates grammar coherence and vocabulary usage to ensure high-quality assessments. By streamlining the grading process, this system aims to provide timely feedback to students while enhancing grading consistency and ultimately improving learning outcomes.
Primary: To streamline the essay grading process for university professors while ensuring fairness, consistency, and prompt feedback for students.
- To develop an automated grading system, reducing the time and effort required for essay assessment.
- Implement NLP algorithms to accurately analyze the content, structure, and coherence of essays.
- Provide timely feedback to students to facilitate their learning process.
- Minimize subjective biases in grading by employing objective NLP-based evaluation techniques.
- Design an intuitive and user-friendly interface for professors to interact with the grading system.
Project Manager - Diya Sharma
Business Analyst - Abhinav
User Experience (UX) Researcher - Abhinav
Quality Assurance (QA) Tester - Aqeel
Documentation Specialist - Faaizah
Project Manager - Mohamed Firas
Team Lead - Prathamesh
Frontend Developer - Sarah & Prathamesh
Backend Developers - Rahul & Prathamesh
Data Scientist - Mujtaba, Rajdeep, Zubair & Rahul
Tester - Manas
DevOps Engineer - Rehman
| Feature | Requirement |
|---|---|
| Core Requirements | |
| User authentication | Mandatory |
| Essay submission supporting text, Word, and PDF formats | Mandatory |
| Rigorous analysis including word count verification, keyword identification, and fact-checking | Mandatory |
| Grammar coherence and vocabulary usage evaluation | Mandatory |
| Automatic grading and grading report generation | Mandatory |
| Emailing users about queries submitted in Contact Forms, and Subscription Plans | Mandatory |
| Subscription and Usability Features | |
| User subscription plans (Basic, Premium) | Mandatory |
| User-friendly interface (Designs by Client Team) | Mandatory |
| Grading report generation with detailed feedback - Payment gateway integration for premium plans | Suggested |
| Upcoming Features | |
| Automated feedback generation | Future Suggestion |
| Integration with learning management systems | Future Suggestion |
| Adaptive learning algorithms for personalized feedback | Future Suggestion |
- Home: Introduce the GradeScale platform and its features.
- About Us: Provide information about the company and its mission.
- Login/Register: User authentication and registration page.
- Contact: Contact form for support inquiries.
- Uploading Page: Essay submission form with upload functionality.
- Submission Page: Confirmation page after essay submission.
- Grading Page: Displays grades and feedback for submitted essays.
- Grading Report: Detailed report on essay analysis and grading.
- Get Pricing: Subscription plans and pricing information.
- Terms and Privacy Policy: Legal information.
- Careers: Information about job opportunities at GradeScale.
Methodologies for Evaluation Criteria:
- Accuracy: Uses a Hugging Face's pre-trained sentence-transformers model (all-MiniLM-L6-v2) to compute the semantic similarity between two text inputs. It converts the input sentences into 384-dimensional vectors using the model, calculates the cosine similarity between these vectors, and then adjusts this raw similarity score into a percentage format with a non-linear transformation.
- Organization: Uses all-MiniLM-L6-v2 to evaluate the organization quality of a given text by measuring the semantic similarity between consecutive sentences. It first splits the input text into sentences, computes their pairwise cosine similarities to generate embeddings, and then averages these similarities to provide an overall organization score expressed as a percentage.
- Grammar: Employs the HappyTransformer and nltk libraries for grammar correction and evaluation. It initializes a HappyTextToText model specifically tuned for grammar correction using a T5 transformer model (vennify/t5-base-grammar-correction). The code also uses NLTK's punkt tokenizer to tokenize text and computes the BLEU score to assess the grammatical accuracy of the corrected text.
- Vocabulary: Uses the NLTK library to evaluate the vocabulary richness of a given text. It starts by tokenizing the text, then lemmatizes each word to its base form using WordNetLemmatizer, and removes common stopwords as well as non-alphanumeric words. The vocabulary score is calculated as the ratio of unique words to the total number of words after processing, providing a measure of the text's lexical diversity.
- Punctuation: Utilizes the deepmultilingualpunctuation library to restore punctuation in text and calculate a punctuation score to evaluate how well the original text matches the punctuation-restored text. A function is defined to count punctuation marks using Python's string.punctuation. It then calculates a punctuation score based on the ratio of original punctuation count to corrected punctuation count. If no punctuation is restored, the score is adjusted based on the number of words in the sentence.
- Relevance: Uses the pre-trained model - 'clips/mfaq' to evaluate the relevance of a student's answer to a specified question to a reference answer. The question, inputted answer, and reference answers are encoded into vector embeddings and their cosine similarities are computed, assessing how closely the student's response aligns with the reference in terms of contextual relevance.
- Coherence: Uses pre-trained model, aisingapore/coherence-momentum to evaluate the coherence of two pieces of text (the reference answer and the student-inputted answer). The texts are preprocessed and converted into tensor formats suitable for the model, then compute a coherence score for each text, returning these scores as an array.
- Framework: Flask
- Database: Sqlite3
- Jinja with Html & CSS.
- GitHub - for collaborative development, version control, and code review
- Digitalocean VPS - Ubuntu 22.04
- Backend Testing: Python's unit test framework for unit testing.
- Frontend Testing: Selenium WebDriver for automated UI testing.
- Sign Up for DigitalOcean: Utilize the free credits available from the GitHub Student Developer Pack to create an account on DigitalOcean.
- Create a Droplet: Set up a new Droplet, which is a Virtual Private Server (VPS), on DigitalOcean. Choose the specifications that match your application needs.
- Install Python and Packages: Access your Droplet via SSH & install all the necessary Python packages required for the application.
- Deploy the Application: Upload the application code to the Droplet. Use the
nohupcommand to run your application code in production mode as a background process. This allows the application to continue running even after you disconnect from the server. - Access the Application: Initially, access your application through the IP address and specific port number provided by the Droplet.
- Future Configuration for Domain Name: Once you acquire a domain name, plan to configure it with Nginx on your Droplet to route traffic to your application. This setup will allow you to access the application through a domain name instead of an IP address.
User Authentication: Test user registration, login, and logout functionality.
Grading Model: Validate grading algorithm accuracy and relevance checking functionality.
Database Management: Verify CRUD operations for user data and essay submissions.
Homepage: Verify layout, navigation links, and content display.
Login/Signup Screens: Validate input fields, form submission, and authentication processes.
Error Handling: Test error messages and validation prompts for invalid inputs.
Designs:
NLP Model: