From dc178b61dcc3464ae1784d1df45d7f053e42fc8b Mon Sep 17 00:00:00 2001 From: Andre Carvalho Date: Wed, 19 Feb 2025 10:25:19 +0000 Subject: [PATCH 1/2] add deepseek to readme --- README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/README.md b/README.md index 4c67a59..fbba5fc 100644 --- a/README.md +++ b/README.md @@ -99,6 +99,17 @@ Here's the steps to set up the project locally: 7. Fill in the models name and check the "active" checkbox in red boxes in the image bellow and click "Save" (pay attention to the LLM and Embeddings model name placement): ![local_models_admin_fields](docs/assets/local_models_admin_fields.png) +### Using DeepSeek R1 with nomic: + +1. `make up` +2. `make createuser` +3. `make ollama model=nomic-embed-text:latest` +4. `make ollama model=deepseek-r1` +5. Go to [http://localhost:8000/admin](http://localhost:8000/admin) in internet browser and login (admin/admin) +6. Select "Models" option in the menu on the left +7. Fill in the models name and check the "active" checkbox in red boxes in the image below and click "Save" (pay attention to the LLM and Embeddings model name placement): + ![local_models_admin_fields](docs/assets/local_models_admin_fields.png) + ## Prompt Examples There are some prompt examples in [here](docs/prompt_examples.md). These are sorted by complexity and are a way for us to measure the quality of the LLM's response. From 45565e30a977f7504a53a56eaa31cc2aad07c69d Mon Sep 17 00:00:00 2001 From: Andre Carvalho Date: Wed, 26 Feb 2025 13:11:14 +0000 Subject: [PATCH 2/2] readme and num_ctx --- README.md | 2 +- labs/llm/ollama.py | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 4250e89..4b71251 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ Here's the steps to set up the project locally: 1. `make up` 2. `make createuser` 3. `make ollama model=nomic-embed-text:latest` -4. `make ollama model=deepseek-r1` +4. `make ollama model=deepseek-r1` | `deepseek-coder` | `deepseek-coder-v2`, coder-v2 is the best if you have a good machine. 5. Go to [http://localhost:8000/admin](http://localhost:8000/admin) in internet browser and login (admin/admin) 6. Select "Models" option in the menu on the left 7. Fill in the models name and check the "active" checkbox in red boxes in the image below and click "Save" (pay attention to the LLM and Embeddings model name placement): diff --git a/labs/llm/ollama.py b/labs/llm/ollama.py index 12eca43..4a2fb57 100644 --- a/labs/llm/ollama.py +++ b/labs/llm/ollama.py @@ -12,6 +12,7 @@ def completion_without_proxy(self, messages, *args, **kwargs): model=self._model_name, messages=messages, format="json", + options={"num_ctx": 8192}, *args, **kwargs, )