feat: Integrate Llama4 and image file support #22
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Integrates a Llama4-compatible model (using ChatGoogleGenerativeAI with a Gemini model as a proxy) for PDF and image querying. Replaces OpenAI dependencies with Google's generative AI and HuggingFace embeddings. Key changes: - Modified `pdfquery.py` to use `ChatGoogleGenerativeAI` and `HuggingFaceEmbeddings`. - Updated API key handling from OpenAI to Google. - Enhanced `pdfquery.py` to process PNG and JPG image files using `UnstructuredImageLoader` and Tesseract OCR for text extraction. - Updated `streamlitui.py` to accept Google API keys and allow uploading of PDF and image files (PNG, JPG, JPEG). - Updated `requirements.txt` to include `langchain-google-genai`, `sentence-transformers`, `pytesseract`, and `Pillow`, and commented out `openai`. - Tesseract OCR engine installation is handled during image processing. Further manual testing is recommended to ensure full functionality of the Streamlit application, API interactions, and OCR accuracy.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Integrates a Llama4-compatible model (using ChatGoogleGenerativeAI with a Gemini model as a proxy) for PDF and image querying. Replaces OpenAI dependencies with Google's generative AI and HuggingFace embeddings.
Key changes:
pdfquery.pyto useChatGoogleGenerativeAIandHuggingFaceEmbeddings.pdfquery.pyto process PNG and JPG image files usingUnstructuredImageLoaderand Tesseract OCR for text extraction.streamlitui.pyto accept Google API keys and allow uploading of PDF and image files (PNG, JPG, JPEG).requirements.txtto includelangchain-google-genai,sentence-transformers,pytesseract, andPillow, and commented outopenai.Further manual testing is recommended to ensure full functionality of the Streamlit application, API interactions, and OCR accuracy.