Skip to content

This project aims to evaluate and compare different Large Language Models (LLMs) for the task of extracting email signature information and structuring it into a JSON format. The evaluated models are OpenAI GPT-3.5 turbo and Anthropic Claude 3. The project includes prompt engineering, testing, and iteration to achieve the best results.

Notifications You must be signed in to change notification settings

edogola4/Email-Signature-Extraction-with-LLMs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Email-Signature-Extraction-with-LLMs

Email Signature Extractor & Evaluator

This project is designed to evaluate the performance of different Large Language Models (LLMs) in extracting email signature information and structuring it into a JSON format.

Project Structure

  • config.py: Contains the OpenAI API key.
  • functions.py: Defines the functions used for prompt evaluation and result analysis.
  • main.py: The main script that runs the evaluation.

Setup

  1. Clone the repository:

    git clone https://github.com/your-repo/email_signature_extractor_evaluator.git
    cd email_signature_extractor_evaluator
  2. Set up a Python virtual environment:

    python3 -m venv .venv
    source .venv/bin/activate
  3. Install the required packages:

    pip install openai transformers
  4. Set your OpenAI API key in config.py:

    # config.py
    OPENAI_API_KEY = "your_openai_api_key_here"

Usage

Run the main script to evaluate the prompt against the test cases:

python main.py


Contact

For any questions or issues, please contact [brandon14ogola@gmail.com].

About

This project aims to evaluate and compare different Large Language Models (LLMs) for the task of extracting email signature information and structuring it into a JSON format. The evaluated models are OpenAI GPT-3.5 turbo and Anthropic Claude 3. The project includes prompt engineering, testing, and iteration to achieve the best results.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •