Added Ollama Support #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

dustinwloring1988 wants to merge 9 commits into kleneway:main from dustinwloring1988:main

.cursor-tasks.md

-Original file line number
+Diff line change
@@ Expand Up / @@ -2,129 +2,6 @@ @@
     ---
-    ### 🚀 **Project Initialization and Setup**
-    - [x] Create a GitHub repository named `forq-cli`.
-    - [x] Clone repository locally and initialize with `npm init -y`.
-    - [x] Set up `.gitignore` to exclude `node_modules`, `.env`, logs, and build artifacts.
-    - [x] Install TypeScript and initialize configuration (`tsconfig.json`).
-    - [x] Configure ESLint (`.eslintrc.json`) and Prettier (`.prettierrc`) for code formatting and linting.
-    - [x] Add scripts to `package.json` for build, lint, test, and run.
-    - [x] Set up basic folder structure (`src/`, `tests/`, `bin/`, `config/`).
-    ---
-    ### 📦 **CLI Interface and Command Parsing**
-    - [x] Install `commander` and set up basic CLI entry point (`bin/forq.ts`).
-    - [x] Implement main command to invoke interactive REPL (`forq repl`).
-    - [x] Implement help command (`forq --help`) that displays available commands.
-    - [x] Ensure executable permissions (`chmod +x bin/forq.ts`) and link via npm scripts.
-    - [x] Verify basic CLI execution (`./bin/forq.ts repl`) runs without errors.
-    ---
-    ### 🎛️ **Interactive REPL Implementation**
-    - [x] Install and configure `readline` or `enquirer` for user input.
-    - [x] Implement interactive REPL loop: prompt user, process input, return response.
-    - [x] Handle basic REPL commands (`/help`, `/clear`, `/exit`) with meaningful output.
-    - [x] Maintain command history navigation using arrow keys.
-    - [x] Add REPL prompt customizations (colors, context indicators).
-    ---
-    ### 🧠 **System Prompt and User Prompt Management**
-    - [x] Create and store the `systemPrompt` text in a dedicated config file (`config/systemPrompt.ts`).
-    - [x] Load and inject `systemPrompt` at initialization of AI context.
-    - [x] Accept user input as a structured `userPrompt` for sending to AI API.
-    - [x] Ensure prompts follow formatting standards (Markdown/structured JSON).
-    - [x] Log user and AI prompts in timestamped conversation history (`logs/conversation.log`).
-    ---
-    ### ⚙️ **AI Integration and Semantic Querying**
-    - [x] Install and configure the Anthropic SDK (or compatible alternative API client).
-    - [x] Set up `.env` file to securely store API keys and environment variables.
-    - [x] Implement function to query AI (`queryAI(messages: Message[])`) with streaming output.
-    - [x] Verify basic API connectivity by sending a simple test message and receiving response.
-    ---
-    ### 📂 **Tool System: Core Implementation**
-    - [x] Define `Tool` interface with structured inputs, outputs, and execution logic.
-    - [x] Implement dynamic tool loading mechanism (e.g., scan `tools/` directory at startup).
-    - [x] Provide a method (`executeTool`) to invoke tools based on AI-generated tool calls.
-    - [x] Verify basic tool invocation through mock calls in REPL.
-    ---
-    ### 🛠️ **Individual Tools Implementation**
-    #### **File System Tools**
-    - [x] Implement `listDir` tool: List files/directories at given path.
-    - [x] Implement `readFile` tool: Return file content securely.
-    - [x] Implement `editFile` tool: Overwrite file content after diff verification.
-    - [x] Implement `deleteFile` tool: Delete specified file safely (with confirmation prompt).
-    - [x] Implement `createFile` tool: Create new files with content, checking for existing files.
-    #### **Search Tools**
-    - [x] Implement `fileSearch` tool: Fuzzy match filenames across directories.
-    - [x] Implement `ripgrepSearch` tool: Regex content search via `ripgrep`.
-    #### **Semantic Tools**
-    - [x] Set up basic semantic embedding mechanism (stub function for embedding text).
-    - [x] Implement `semanticSearch` tool: Return semantically relevant code snippets based on query.
-    - [x] Implement `readSemanticSearchFiles` tool: Retrieve top semantic matches with full file content.
-    #### **Terminal Command Tool (Bash Integration)**
-    - [x] Implement `bash` tool: Execute commands in a secure, persistent shell session.
-    - [x] Ensure commands have strict permission verification (banned commands, safe execution environment).
-    - [x] Capture and handle stdout/stderr, enforce timeout mechanisms (default 2 min).
-    - [x] Persist environment variables and working directories between bash calls.
-    ---
-    ### 🛡️ **Security & Permission System**
-    - [x] Implement per-session permission store (in-memory).
-    - [x] Prompt user clearly for tool permissions on first use (e.g., file access, shell command execution).
-    - [x] Persist granted permissions in a session configuration (`config/session-permissions.json`).
-    - [x] Enforce strict checking of permissions before any sensitive tool action.
-    ---
-    ### 📚 **Context Management**
-    - [x] Implement automatic loading of project-specific instructions from `FORQ.md`.
-    - [x] Collect git context (current branch, modified files, recent commits) using bash tool.
-    - [x] Provide summarized directory structure to AI on session start.
-    - [x] Compact conversation history periodically (`/compact` command implementation).
-    ---
-    ### 📑 **Configuration Management**
-    - [x] Implement global config storage (`~/.forqrc.json`) to persist user preferences and API keys.
-    - [x] Implement project-specific config (`.forqrc.json`) for allowed tools, commands, and local overrides.
-    - [x] Provide CLI command (`forq config`) to view and edit configurations.
-    ---
-    ### 📊 **Analytics & Error Handling**
-    - [x] Implement basic analytics logging (session duration, commands used).
-    - [x] Implement local error logging (`logs/error.log`) capturing stack traces.
-    ---
     ### 🔍 **Logging & Auditability**
     - [x] Ensure each AI action (tool execution, API call) is logged in structured logs (`logs/actions.log`).
@@ Expand All / @@ -148,3 +25,66 @@ @@
     - [x] Add detailed documentation (`docs/`) for each CLI command, prompt conventions, and available tools.
     - [x] Provide example workflows (e.g., bugfixes, feature implementations) in the documentation.
     - [x] Implement `forq help <command>` providing detailed contextual help.
+    ---
+    ### 🔄 **Implment Self Host Mode with ollama**
+    #### **1. Setup Ollama Integration**
+    a) **Update Configuration**
+    - [x] Add Ollama section to `ForqConfig` in `src/utils/config.ts`
+    - [x] Add default Ollama settings to `createDefaultConfig`
+    - [x] Update `.forqrc.json.example` with Ollama configuration
+    b) **Ollama API Client**
+    - [x] Create `src/api/ollama.ts` for Ollama API integration
+    - [x] Implement basic API functions:
+      - [x] `listModels()`
+      - [x] `createCompletion()`
+      - [x] `createEmbedding()`
+    - [x] Add error handling and response type definitions
+    c) **Model Management**
+    - [x] Implement model pulling if not present
+    - [x] Add model verification before usage
+    - [x] Add model status checking
+    #### **2. Embedding Model Integration**
+    a) **Model Configuration**
+    - [x] Add snowflake-arctic-embed2 model settings
+    - [x] Implement model download/verification
+    - [x] Add embedding model configuration options
+    b) **Embedding Generation**
+    - [x] Create `src/embeddings/ollama.ts` for embedding functionality
+    - [x] Implement embedding generation using snowflake-arctic-embed2
+    - [x] Add caching for generated embeddings
+    #### **3. Self Mode Implementation**
+    a) **Semantic Search Updates**
+    - [x] Modify semantic search to use local embeddings
+    - [x] Update vector similarity calculations
+    - [x] Add performance optimizations for local processing
+    b) **Command Setup**
+    - [x] Add `self` command to CLI options
+    - [x] Create `src/modes/self.ts` for self mode implementation
+    - [x] Implement command parsing and validation
+    c) **Message Handling & Response Processing**
+    - [x] Create Ollama-specific message formatter
+    - [x] Implement conversation history management
+    - [x] Add context window management
+    - [x] Implement streaming response handling
+    - [x] Implement proper error handling
+    d) **Integration**
+    - [x] Connect all components
+    - [x] Add proper logging
+    - [x] Run the app starting into self mode
+    e) **Graceful Fallbacks**
+    - [x] Implement graceful fallbacks

.cursor-updates

-Original file line number
+Diff line change
@@ Expand Up / @@ -72,3 +72,32 @@ @@
     - Implemented a robust promise-based permission system that properly awaits user confirmation before proceeding with tool execution.
     - Fixed critical bug in tool permission handling that was causing the app to crash when requesting permission for createFile operations.
+    Added detailed breakdown of Ollama integration task into three main sections:
+. Setup Ollama Integration (configuration, API client, model management)
+. Embedding Model Integration (model config, embedding generation, semantic search updates)
+. Self Mode Implementation (command setup, message handling, response processing, integration)
+    - Added Ollama configuration to ForqConfig interface and default settings
+    - Added Ollama API client with model management, completion, and embedding functionality
+    ## 2024-03-14
+    - Implemented Self Mode Command Setup with `forq self` command to enable local Ollama-based operation.
+    - Created `src/modes/self.ts` with complete command parsing, validation, and integration with Ollama client.
+    - Added special command handling (/help, /clear, /exit, /compact, /status) for self-hosted mode.
+    - Implemented tool call parsing and validation to enable tool usage with local Ollama models.
+    - Updated CLI help documentation with detailed description of self mode features and usage examples.
+    - Completed self mode implementation with proper logging and graceful fallbacks
+    - Added ping method to OllamaClient for server availability check
+    - Added retry mechanism with exponential backoff for streaming responses
+    - Added timeout handling for streaming responses
+    - Added proper error handling and fallbacks for model loading and streaming
+    - Updated tasks checklist to mark completed items
+    - Added import statement for `axios` in `src/modes/self.ts` to resolve 'axios is not defined' error.
+    - Added MCP (Message Control Protocol) server support with WebSocket integration for external client connections
+    - Fixed MCP server to run without requiring ANTHROPIC_API_KEY by making API key validation conditional
+    - Added built-in math server to MCP with support for basic arithmetic operations (add, subtract, multiply, divide) through WebSocket messages

.forq/permissions.json

-Original file line number
+Diff line change
@@ -1,59 +1,3 @@
     {
-      "tools": {
-        "readFile": [
-          {
-            "type": "file_system",
-            "granted": true,
-            "timestamp": 1741122828304
-          }
-        ],
-        "editFile": [
-          {
-            "type": "file_system",
-            "granted": true,
-            "timestamp": 1741123004505
-          }
-        ],
-        "createFile": [
-          {
-            "type": "file_system",
-            "granted": true,
-            "scope": "test.txt",
-            "timestamp": 1741131852486
-          },
-          {
-            "type": "file_system",
-            "granted": true,
-            "scope": "hello.js",
-            "timestamp": 1741148444696
-          },
-          {
-            "type": "file_system",
-            "granted": true,
-            "scope": "tests/utils/logger.test.ts",
-            "timestamp": 1741219199931
-          },
-          {
-            "type": "file_system",
-            "granted": true,
-            "scope": "tests/utils/config.test.ts",
-            "timestamp": 1741219249495
-          }
-        ],
-        "deleteFile": [
-          {
-            "type": "file_system",
-            "granted": true,
-            "scope": "./test.txt",
-            "timestamp": 1741132947992
-          }
-        ],
-        "bash": [
-          {
-            "type": "shell_command",
-            "granted": true,
-            "timestamp": 1741133287243
-          }
-        ]
-      }
+      "tools": {}
     }

.forqrc.json

-Original file line number
+Diff line change
@@ -0,0 +1,36 @@
+    {
+      "api": {
+        "anthropic": {
+          "apiKey": "your-api-key-here",
+          "model": "claude-3-opus-20240229",
+          "maxTokens": 4000,
+          "temperature": 0.7,
+          "completeToolCycle": true
+        },
+        "openai": {
+          "apiKey": "your-api-key-here",
+          "model": "gpt-4-turbo-preview"
+        },
+        "ollama": {
+          "host": "http://localhost",
+          "port": 11434,
+          "model": "llama3.1",
+          "embeddingModel": "snowflake-arctic-embed2",
+          "maxTokens": 4096,
+          "temperature": 0.7,
+          "contextWindow": 8192,
+          "systemPrompt": "You are a helpful AI assistant."
+        }
+      },
+      "tools": {},
+      "repl": {
+        "historySize": 100,
+        "autoCompactThreshold": 40
+      },
+      "logging": {
+        "level": "info",
+        "logConversation": true,
+        "logToolCalls": true,
+        "logPerformance": false
+      }
+    }

.forqrc.json.example

-Original file line number
+Diff line change
@@ -0,0 +1,36 @@
+    {
+      "api": {
+        "anthropic": {
+          "apiKey": "your-api-key-here",
+          "model": "claude-3-opus-20240229",
+          "maxTokens": 4000,
+          "temperature": 0.7,
+          "completeToolCycle": true
+        },
+        "openai": {
+          "apiKey": "your-api-key-here",
+          "model": "gpt-4-turbo-preview"
+        },
+        "ollama": {
+          "host": "http://localhost",
+          "port": 11434,
+          "model": "mistral",
+          "embeddingModel": "nomic-embed-text",
+          "maxTokens": 4096,
+          "temperature": 0.7,
+          "contextWindow": 8192,
+          "systemPrompt": "You are a helpful AI assistant."
+        }
+      },
+      "tools": {},
+      "repl": {
+        "historySize": 100,
+        "autoCompactThreshold": 40
+      },
+      "logging": {
+        "level": "info",
+        "logConversation": true,
+        "logToolCalls": true,
+        "logPerformance": false
+      }
+    }

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added Ollama Support #2

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!