Add proxy-lite and instructor_ex integration

.gitmodules

@@ -0,0 +1 @@
+

INTEGRATION_README.md

@@ -0,0 +1,189 @@
+# Proxy-lite + Instructor_ex Integration
+
+This repository contains an integration between `proxy-lite` (a FastAPI-based web automation service) and `instructor_ex` (an Elixir port of the Instructor library for structured data extraction).
+
+## Overview
+
+The integration allows for:
+1. Web automation tasks using Playwright via the proxy-lite service
+2. Structured data extraction from web content using instructor_ex and a local LLM
+3. Composable functions that combine these capabilities for various use cases
+
+## Components
+
+- **proxy_service.py**: A FastAPI service that provides web automation functionality
+- **proxy_client.ex**: An Elixir client for interacting with the proxy-lite service
+- **proxy_schemas.ex**: Ecto schemas for structured data extraction
+- **proxy_instructor.ex**: Integration logic combining proxy-lite and instructor_ex
+
+## Setup
+
+### Prerequisites
+
+- Python 3.10+ with FastAPI and Playwright
+- Elixir 1.14+
+- A running llama.cpp server with a compatible model
+
+### Starting the Services
+
+1. Start the llama.cpp server:
+   ```
+   ./start_llama_server.sh /path/to/model.gguf
+   ```
+
+2. Start the proxy-lite service:
+   ```
+   ./start_proxy_service.sh
+   ```
+
+## Usage
+
+Run the example script to see the integration in action:
+
+```
+./example_usage.exs
+```
+
+The script demonstrates searching for weather information and extracting structured data from the results.
+
+## Key Features
+
+- **Task Automation**: Perform complex web tasks like searching, navigation, and data extraction
+- **Structured Data Extraction**: Extract typed data from web content using LLM capabilities
+- **Error Handling**: Robust handling of timeouts and failures during web automation
+- **Configurable Options**: Control browser visibility, timeout duration, and more
+
+## Implementation Details
+
+The integration follows these steps:
+1. proxy_client.ex sends a task request to the proxy-lite service
+2. proxy_service.py executes the task using Playwright
+3. The results (including HTML and screenshots) are returned to the client
+4. proxy_instructor.ex uses instructor_ex to extract structured data
+5. The final results are presented with complete details and formatting
+
+## Recent Improvements
+
+- Enhanced error handling and timeout management
+- Improved extraction of complete answers from automation results
+- Better prompt engineering for the LLM to ensure complete information
+- Fixed variable warnings and improved code organization
+
+## Architecture
+
+The integration uses a service-oriented architecture:
+
+- **Python Service**: A FastAPI server that exposes proxy-lite functionality
+- **Elixir Client**: Modules for instructor_ex to communicate with the service
+- **Ecto Schemas**: Well-defined structures for extracting data from web pages
+
+## Prerequisites
+
+- Python 3.11+
+- Elixir 1.14+
+- llama.cpp installed with a compatible model
+- Playwright installed
+
+## Setup Instructions
+
+### 1. Set Up llama.cpp
+
+First, make sure you have llama.cpp installed and running:
+
+```bash
+# Install llama.cpp
+brew install llama.cpp
+
+# Start the server with a suitable model
+llama-server --port 8080 -ngl 999 -hf Qwen/Qwen2.5-7B-Instruct-GGUF
+```
+
+### 2. Set Up the proxy-lite Service
+
+Make the service script executable and start it:
+
+```bash
+chmod +x start_proxy_service.sh
+./start_proxy_service.sh
+```
+
+This will:
+- Install necessary Python dependencies
+- Start the proxy-lite service on port 8000
+
+### 3. Run the Example Script
+
+Make the example script executable and run it:
+
+```bash
+chmod +x example_usage.exs
+./example_usage.exs
+```
+
+## Using the Integration in Your Code
+
+### Simple Usage Example
+
+```elixir
+# Configure your LLM adapter
+config = [
+  adapter: Instructor.Adapters.Llamacpp,
+  api_url: "http://localhost:8080"
+]
+
+# Perform a web search and get structured results
+{:ok, results} = ProxyInstructor.search_web("Best restaurants in San Francisco")
+
+# Extract entities from a specific webpage
+{:ok, page_data} = ProxyInstructor.extract_page_info("https://en.wikipedia.org/wiki/San_Francisco")
+
+# Perform a more complex task with automated browsing
+{:ok, summary} = ProxyInstructor.perform_task("Find the current weather in New York City")
+```
+
+## Integration Files
+
+- `proxy_service.py`: FastAPI service for proxy-lite
+- `start_proxy_service.sh`: Script to start the service
+- `proxy_client.ex`: Elixir client for the service API
+- `proxy_schemas.ex`: Ecto schemas for structured data
+- `proxy_instructor.ex`: Main integration module
+- `example_usage.exs`: Example script
+
+## Best Practices for Production Use
+
+1. **Resource Management**: Properly manage browser instances
+2. **Error Handling**: Implement retries and timeouts
+3. **Security**: Be careful with user-provided inputs
+4. **Performance**: Consider caching results for repeated queries
+5. **Isolation**: Run the Python service in a container
+
+## Customization
+
+You can customize this integration by:
+
+1. Extending the schemas in `proxy_schemas.ex`
+2. Adding new functions to `proxy_instructor.ex`
+3. Configuring different models for llama.cpp
+4. Modifying the proxy-lite configuration
+
+## Troubleshooting
+
+### Service Not Starting
+
+- Check if the required ports (8000 for the service, 8080 for llama.cpp) are available
+- Ensure all dependencies are installed correctly
+
+### Browser Issues
+
+- If you encounter "browser executable not found" errors, run `playwright install` manually
+- For headless mode issues on Mac, try running with `headless: false`
+
+### Model Performance
+
+- For better results, use larger models with llama.cpp (if your hardware supports it)
+- Balance between model size and performance based on your needs
+
+## License
+
+This integration is provided under the MIT license. 

example_usage.exs

@@ -0,0 +1,98 @@
+#!/usr/bin/env elixir
+
+# First, ensure dependencies are available
+Mix.install([
+  {:instructor, path: "instructor_ex"},
+  {:httpoison, "~> 2.0"},
+  {:jason, "~> 1.4"},
+  {:ecto, "~> 3.12"}
+])
+
+# Load our integration modules
+Code.require_file("proxy_client.ex")
+Code.require_file("proxy_schemas.ex")
+Code.require_file("proxy_instructor.ex")
+
+IO.puts("========================================================")
+IO.puts("  Proxy-lite + instructor_ex Integration Example")
+IO.puts("========================================================")
+
+# Check if the llama.cpp server is running
+llama_server_url = "http://localhost:8090"
+
+llama_server_running = try do
+  HTTPoison.get!(llama_server_url <> "/v1/models", [], [recv_timeout: 5000])
+  true
+rescue
+  _ -> false
+end
+
+unless llama_server_running do
+  IO.puts("\n❌ ERROR: llama.cpp server is not running at #{llama_server_url}")
+  IO.puts("Please start the server with:")
+  IO.puts("  llama-server --port 8090 -ngl 1 -m /Users/speed/Library/Caches/llama.cpp/Qwen_Qwen2.5-7B-Instruct-GGUF_qwen2.5-7b-instruct-q2_k.gguf")
+  System.halt(1)
+end
+
+# Check if the proxy-lite service is running
+proxy_service_running = ProxyClient.available?()
+
+unless proxy_service_running do
+  IO.puts("\n❌ ERROR: proxy-lite service is not running")
+  IO.puts("Please start it with:")
+  IO.puts("  ./start_proxy_service.sh")
+  System.halt(1)
+end
+
+IO.puts("\n✅ Services are running!")
+
+# Define some example tasks
+tasks = [
+  "Find the current weather in San Francisco",
+  "Search for the top 3 Italian restaurants in New York City",
+  "Get the latest news about artificial intelligence"
+]
+
+# Ask the user to select a task
+IO.puts("\nPlease select a task to perform:")
+
+Enum.with_index(tasks, 1)
+|> Enum.each(fn {task, index} -> 
+  IO.puts("  #{index}. #{task}")
+end)
+
+IO.puts("  #{length(tasks) + 1}. Custom task (enter your own)")
+
+selected_task_input = IO.gets("\nEnter task number: ") |> String.trim()
+{selected_index, _} = Integer.parse(selected_task_input)
+
+task = if selected_index <= length(tasks) do
+  Enum.at(tasks, selected_index - 1)
+else
+  IO.puts("\nEnter your custom task:")
+  IO.gets("") |> String.trim()
+end
+
+# Configure options
+headless = false
+IO.puts("\nRunning with task: \"#{task}\"")
+IO.puts(if headless, do: "Browser will run in headless mode", else: "Browser will be visible")
+
+# Execute the task
+IO.puts("\n🚀 Executing task...")
+
+case ProxyInstructor.perform_task(task, [headless: headless]) do
+  {:ok, result} ->
+    IO.puts("\n✅ Task completed!")
+    IO.puts("\nSummary: #{result.summary}")
+
+    if result.answer && result.answer != "" do
+      IO.puts("\nAnswer: #{result.answer}")
+    end
+
+    steps = result.steps_taken || 0
+    IO.puts("\nSteps taken: #{steps}")
+
+  {:error, reason} ->
+    IO.puts("\n❌ ERROR: #{reason}")
+end 

proxy_client.ex

@@ -0,0 +1,91 @@
+defmodule ProxyClient do
+  @moduledoc """
+  Client for interacting with the proxy-lite service.
+
+  This module provides functions to send requests to the proxy-lite service
+  which handles web automation via Playwright.
+  """
+
+  @proxy_service_url "http://localhost:8001"
+
+  @doc """
+  Performs a web automation task using proxy-lite and returns the results.
+
+  ## Parameters
+
+  - `query`: The task to perform (e.g., "Find the weather in New York")
+  - `opts`: Additional options for the task
+    - `:homepage` - The starting URL (default: "https://www.google.com")
+    - `:headless` - Whether to run in headless mode (default: false)
+    - `:include_html` - Whether to include HTML content in response (default: true)
+    - `:annotate_image` - Whether to annotate screenshots (default: true)
+    - `:max_steps` - Maximum number of steps to take (default: 50)
+
+  ## Returns
+
+  {:ok, result} on success, {:error, reason} on failure
+
+  ## Examples
+
+      iex> ProxyClient.run_task("Find the top 3 Italian restaurants in San Francisco")
+      {:ok, %{
+        "status" => "success",
+        "screenshots" => [...],
+        "html_content" => "...",
+        "results" => %{...}
+      }}
+  """
+  def run_task(query, opts \\ []) do
+    # Get options with defaults
+    homepage = Keyword.get(opts, :homepage, "https://www.google.com")
+    headless = Keyword.get(opts, :headless, false)
+    include_html = Keyword.get(opts, :include_html, true)
+    annotate_image = Keyword.get(opts, :annotate_image, true)
+    max_steps = Keyword.get(opts, :max_steps, 50)
+
+    # Prepare the request body
+    body = %{
+      query: query,
+      homepage: homepage,
+      headless: headless,
+      include_html: include_html,
+      annotate_image: annotate_image,
+      max_steps: max_steps
+    }
+
+    # Make the HTTP request
+    case HTTPoison.post(
+      "#{@proxy_service_url}/run",
+      Jason.encode!(body),
+      [{"Content-Type", "application/json"}],
+      recv_timeout: 300_000  # 5-minute timeout
+    ) do
+      {:ok, %HTTPoison.Response{status_code: 200, body: response_body}} ->
+        # Parse the response
+        case Jason.decode(response_body) do
+          {:ok, parsed} -> {:ok, parsed}
+          {:error, _} = error -> error
+        end
+
+      {:ok, %HTTPoison.Response{status_code: code, body: body}} ->
+        {:error, "HTTP Error #{code}: #{body}"}
+
+      {:error, %HTTPoison.Error{reason: reason}} ->
+        {:error, "HTTP Request Failed: #{inspect(reason)}"}
+    end
+  end
+
+  @doc """
+  Checks if the proxy-lite service is available.
+
+  ## Returns
+
+  true if the service is available, false otherwise
+  """
+  def available? do
+    case HTTPoison.get("#{@proxy_service_url}/") do
+      {:ok, %HTTPoison.Response{status_code: 200}} -> true
+      _ -> false
+    end
+  end
+end