Skip to content

Conversation

@erkinalp
Copy link
Contributor

@erkinalp erkinalp commented Dec 16, 2024

Alternative Model Support with Template Segmentation

This PR adds support for alternative language models and improves paper generation capabilities through template segmentation:

New Models Added

  • Gemini Pro (via Google Cloud)
  • Grok-1 (via xAI)
  • LLaMa 3.3 (both API and local variants)
  • LLaMa 3.2 1B and LLaMa 3.1 8B (local variants with template segmentation)

Template Segmentation Integration

  • Cherry-picked changes from PR The segmented paper template allows weak LLMs to write articles #114 for segmented paper templates
  • Added model-specific edit format handling:
    • "whole" mode for resource-constrained models (llama3.1:8b, llama3.2:1b)
    • "diff" mode for stronger models
  • Verified that template segmentation makes llama3.1:8b and llama3.2:1b viable for paper generation

Implementation Details

  • Added OpenAI-compatible local model support via Ollama
  • Implemented model-specific response handling
  • Added environment variable validation for API credentials
  • Updated documentation with model performance tiers

Testing

  • Verified pipeline integration with local models
  • Confirmed template segmentation functionality
  • Tested basic response generation
  • Validated edit format handling

Link to Devin run: https://app.devin.ai/sessions/2ec43d6fe7a84849a348753167e5a895

- Add new model identifiers to AVAILABLE_LLMS
- Extend create_client for new model support
- Add environment variable validation
- Implement model-specific response handling
- Update requirements.txt with google-cloud-aiplatform

Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
@erkinalp erkinalp marked this pull request as draft December 16, 2024 13:21
devin-ai-integration bot and others added 2 commits December 16, 2024 13:47
Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
@erkinalp erkinalp changed the title feat: Add support for Gemini Pro, Grok-1, and LLaMa 3.3 models feat: Add alternative model support including Gemini Pro, Grok-1, and LLaMa variants Dec 16, 2024
@erkinalp erkinalp marked this pull request as ready for review December 16, 2024 14:05
vasiliyeskin and others added 5 commits December 16, 2024 14:29
It is quite difficult for weak models to do so based on the default template, as one error in replaced text can lead to termination of editing in the *SEARCH/REPLACE mode*. Splitting the text into segments and enabling "whole" mode editing helps solve this problem.

The approach was tested on the "2d_diffusion" task using the "groq/llama3-8b-8192".
…n with segmented templates

Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
…on support

Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
…rs, and template compatibility

Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
- Add retry limit and better error handling in extract_json_between_markers
- Replace assert statements with try-catch blocks across all files
- Add proper error messages and recovery mechanisms
- Prevent infinite loops when JSON extraction fails

Fixes SakanaAI#154

Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
@erkinalp erkinalp closed this Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants