diff --git a/DESCRIPTION b/DESCRIPTION index 2deeaf6..6260b6e 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -16,7 +16,7 @@ BugReports: https://github.com/mlverse/chattr/issues License: MIT + file LICENSE Encoding: UTF-8 Roxygen: list(markdown = TRUE) -RoxygenNote: 7.3.1 +RoxygenNote: 7.3.2 Imports: rstudioapi, lifecycle, diff --git a/NEWS.md b/NEWS.md index eab38c9..e7b4da1 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,5 +1,6 @@ # chattr (dev) +* Updating support for Databricks to use newer Meta Llama models * Prevents OpenAI 4o from showing as an option if no token is found # chattr 0.2.0 diff --git a/R/chattr-use.R b/R/chattr-use.R index 0eb6293..71335f2 100644 --- a/R/chattr-use.R +++ b/R/chattr-use.R @@ -1,7 +1,7 @@ #' Sets the LLM model to use in your session #' @param x The label of the LLM model to use, or the path of a valid YAML #' default file . Valid values are 'copilot', 'gpt4', 'gpt35', 'llamagpt', -#' 'databricks-dbrx', 'databricks-meta-llama3-70b', and 'databricks-mixtral8x7b'. +#' 'databricks-meta-llama-3-1-405b-instruct', and 'databricks-meta-llama-3-3-70b-instruct' #' The value 'test' is also acceptable, but it is meant for package examples, #' and internal testing. #' @param ... Default values to modify. @@ -97,9 +97,8 @@ ch_get_ymls <- function(menu = TRUE) { } if (!dbrx_exists) { - prep_files$`databricks-dbrx` <- NULL - prep_files$`databricks-meta-llama3-70b` <- NULL - prep_files$`databricks-mixtral8x7b` <- NULL + prep_files$`databricks-meta-llama-3-1-405b` <- NULL + prep_files$`databricks-meta-llama-3-3-70b` <- NULL } if (!llama_exists) { diff --git a/README.md b/README.md index c64955a..a509c97 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,9 @@ +--- +editor_options: + markdown: + wrap: 72 +--- + # chattr @@ -10,6 +16,7 @@ status](https://www.r-pkg.org/badges/version/chattr.png)](https://CRAN.R-project [![](man/figures/lifecycle-experimental.svg)](https://lifecycle.r-lib.org/articles/stages.html#experimental) + - [Intro](#intro) @@ -25,7 +32,7 @@ status](https://www.r-pkg.org/badges/version/chattr.png)](https://CRAN.R-project -## Intro +## Intro {#intro} `chattr` is an interface to LLMs (Large Language Models). It enables interaction with the model directly from the RStudio IDE. `chattr` @@ -37,7 +44,7 @@ tasks. The additional information appended to your request, provides a sort of “guard rails”, so that the packages and techniques we usually recommend as best practice, are used in the model’s responses. -## Install +## Install {#install} Since this is a very early version of the package install the package from GitHub: @@ -46,74 +53,56 @@ from GitHub: remotes::install_github("mlverse/chattr") ``` -## Available models +## Available models {#available-models} `chattr` provides two main integration with two main LLM back-ends. Each back-end provides access to multiple LLM types. The plan is to add more back-ends as time goes by: - ----- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ProviderModelsSetup Instructions
OpenAIGPT Models accessible via the OpenAI’s -REST API. chattr provides a convenient way to interact with -GPT 4, and 3.5.Interact -with OpenAI GPT models
LLamaGPT-ChatLLM models available in your computer. -Including GPT-J, LLaMA, and MPT. Tested on a GPT4ALL model. -LLamaGPT-Chat is a command line chat program for models -written in C++.Interact -with local models
GitHub -CopilotAI pair programmer that offers -autocomplete-style suggestions as you codeInteract -with GitHub Copilot Chat
DatabricksDBRX, Meta Llama 3 70B, and Mixtral 8x7B -via Databricks -foundational model REST API.Interact -with Databricks foundation chat models
- -## Using - -### The App ++------------------+-------------------------------+----------------+ +| Provider | Models | Setup | +| | | Instructions | ++:================:+:=============================:+:==============:+ +| OpenAI | GPT Models accessible via the | [Interact with | +| | OpenAI’s REST API. `chattr` | OpenAI GPT | +| | provides a convenient way to | models](ht | +| | interact with GPT 4, and 3.5. | tps://mlverse. | +| | | github.io/chat | +| | | tr/articles/op | +| | | enai-gpt.html) | ++------------------+-------------------------------+----------------+ +| [LLamaGPT | LLM models available in your | [Interact with | +| -Chat](https://g | computer. Including GPT-J, | local | +| ithub.com/kuvaus | LLaMA, and MPT. Tested on a | mo | +| /LlamaGPTJ-chat) | [GPT4ALL](h | dels](https:// | +| | ttps://gpt4all.io/index.html) | mlverse.github | +| | model. **LLamaGPT-Chat** is a | .io/chattr/art | +| | command line chat program for | icles/backend- | +| | models written in C++. | llamagpt.html) | ++------------------+-------------------------------+----------------+ +| [GitHub | AI pair programmer that | [Interact with | +| Copil | offers autocomplete-style | GitHub Copilot | +| ot](https://docs | suggestions as you code | Chat](http | +| .posit.co/ide/us | | s://mlverse.gi | +| er/ide/guide/too | | thub.io/chattr | +| ls/copilot.html) | | /articles/copi | +| | | lot-chat.html) | ++------------------+-------------------------------+----------------+ +| [Databricks | Meta Llama 3.1 405B and 3.3 | [Interact with | +| ](https://docs.d | 70B via [Databricks | Databricks | +| atabricks.com/en | foundational model REST | foundation | +| /machine-learnin | API | chat | +| g/foundation-mod | ](https://docs.databricks.com | mode | +| els/index.html#d | /en/machine-learning/foundati | ls](https://ml | +| atabricks-founda | on-models/index.html#pay-per- | verse.github.i | +| tion-model-apis) | token-foundation-model-apis). | o/chattr/artic | +| | | les/backend-da | +| | | tabricks.html) | ++------------------+-------------------------------+----------------+ + +## Using {#using} + +### The App {#the-app} The main way to use `chattr` is through the Shiny Gadget app. By default, in RStudio the app will run inside the Viewer pane. `chattr` @@ -192,13 +181,13 @@ The screen that opens will contain the following: ![Screenshot of the Sniny gadget options](man/figures/readme/chat2.png) -### Additional ways to interact +### Additional ways to interact {#additional-ways-to-interact} Apart from the Shiny app, `chattr` provides two more ways to interact with the LLM. For details, see: [Other interfaces](https://mlverse.github.io/chattr/articles/other-interfaces.html) -## How it works +## How it works {#how-it-works} `chattr` enriches your request with additional instructions, name and structure of data frames currently in your environment, the path for the @@ -255,7 +244,7 @@ chattr(preview = TRUE) #> [Your future prompt goes here] ``` -## Keyboard Shortcut +## Keyboard Shortcut {#keyboard-shortcut} The best way to access `chattr`’s app is by setting up a keyboard shortcut for it. This package includes an RStudio Addin that gives us @@ -264,23 +253,29 @@ to be assigned to the addin. The name of the addin is: “Open Chat”. If you are not familiar with how to assign a keyboard shortcut see the next section. -### How to setup the keyboard shortcut +### How to setup the keyboard shortcut {#how-to-setup-the-keyboard-shortcut} - Select *Tools* in the top menu, and then select *Modify Keyboard Shortcuts* + ```{=html} Screenshot that shows where to find the option to modify the keyboard shortcuts + ``` - Search for the `chattr` adding by writing “open chat”, in the search box + ```{=html} Screenshot that shows where to input the addin search + ``` - To select a key combination for your shortcut, click on the Shortcut box and then type *press* the key combination in your keyboard. In my case, I chose *Ctrl+Shift+C* + ```{=html} Screenshot that shows what the interface looks like when a shortcut has been selected + ``` diff --git a/chattr.Rproj b/chattr.Rproj index 53806de..60e08d9 100644 --- a/chattr.Rproj +++ b/chattr.Rproj @@ -1,4 +1,5 @@ Version: 1.0 +ProjectId: 539355de-b8e8-4dff-931c-49b5461c9f07 RestoreWorkspace: Default SaveWorkspace: Default diff --git a/inst/configs/databricks-meta-llama3-70b.yml b/inst/configs/databricks-meta-llama-3-1-405b.yml similarity index 91% rename from inst/configs/databricks-meta-llama3-70b.yml rename to inst/configs/databricks-meta-llama-3-1-405b.yml index 5950b13..931af8a 100644 --- a/inst/configs/databricks-meta-llama3-70b.yml +++ b/inst/configs/databricks-meta-llama-3-1-405b.yml @@ -3,8 +3,8 @@ default: {readLines(system.file('prompt/base.txt', package = 'chattr'))} provider: Databricks path: serving-endpoints - model: databricks-meta-llama-3-70b-instruct - label: Meta Llama 3 70B (Databricks) + model: databricks-meta-llama-3-1-405b-instruct + label: Meta Llama 3.1 405B (Databricks) max_data_files: 0 max_data_frames: 0 include_doc_contents: FALSE diff --git a/inst/configs/databricks-dbrx.yml b/inst/configs/databricks-meta-llama-3-3-70b.yml similarity index 91% rename from inst/configs/databricks-dbrx.yml rename to inst/configs/databricks-meta-llama-3-3-70b.yml index 7cb272a..16e686b 100644 --- a/inst/configs/databricks-dbrx.yml +++ b/inst/configs/databricks-meta-llama-3-3-70b.yml @@ -3,8 +3,8 @@ default: {readLines(system.file('prompt/base.txt', package = 'chattr'))} provider: Databricks path: serving-endpoints - model: databricks-dbrx-instruct - label: DBRX (Databricks) + model: databricks-meta-llama-3-3-70b-instruct + label: Meta Llama 3.3 70B (Databricks) max_data_files: 0 max_data_frames: 0 include_doc_contents: FALSE diff --git a/inst/configs/databricks-mixtral8x7b.yml b/inst/configs/databricks-mixtral8x7b.yml deleted file mode 100644 index 600e1fb..0000000 --- a/inst/configs/databricks-mixtral8x7b.yml +++ /dev/null @@ -1,33 +0,0 @@ -default: - prompt: | - {readLines(system.file('prompt/base.txt', package = 'chattr'))} - provider: Databricks - path: serving-endpoints - model: databricks-mixtral-8x7b-instruct - label: Mixtral 8x7b (Datbricks) - max_data_files: 0 - max_data_frames: 0 - include_doc_contents: FALSE - include_history: TRUE - system_msg: You are a helpful coding assistant - model_arguments: - temperature: 0.01 - max_tokens: 1000 - stream: TRUE -chat: - prompt: | - {readLines(system.file('prompt/base.txt', package = 'chattr'))} - For code output, use RMarkdown code chunks - Avoid all code chunk options -console: - prompt: | - {readLines(system.file('prompt/base.txt', package = 'chattr'))} - For any line that is not code, prefix with a: # - Keep each line of explanations to no more than 80 characters - DO NOT use Markdown for the code -script: - prompt: | - {readLines(system.file('prompt/base.txt', package = 'chattr'))} - For any line that is not code, prefix with a: # - Keep each line of explanations to no more than 80 characters - DO NOT use Markdown for the code diff --git a/man/chattr_use.Rd b/man/chattr_use.Rd index b9f1893..566d551 100644 --- a/man/chattr_use.Rd +++ b/man/chattr_use.Rd @@ -9,7 +9,7 @@ chattr_use(x = NULL, ...) \arguments{ \item{x}{The label of the LLM model to use, or the path of a valid YAML default file . Valid values are 'copilot', 'gpt4', 'gpt35', 'llamagpt', -'databricks-dbrx', 'databricks-meta-llama3-70b', and 'databricks-mixtral8x7b'. +'databricks-meta-llama-3-1-405b-instruct', and 'databricks-meta-llama-3-3-70b-instruct' The value 'test' is also acceptable, but it is meant for package examples, and internal testing.} diff --git a/tests/testthat/_snaps/backend-databricks.md b/tests/testthat/_snaps/backend-databricks.md index 48e13e3..53ef507 100644 --- a/tests/testthat/_snaps/backend-databricks.md +++ b/tests/testthat/_snaps/backend-databricks.md @@ -49,8 +49,8 @@ Message * Provider: Databricks * Path/URL: serving-endpoints - * Model: databricks-meta-llama-3-70b-instruct - * Label: Meta Llama 3 70B (Databricks) + * Model: databricks-meta-llama-3-3-70b-instruct + * Label: Meta Llama 3.3 70B (Databricks) ! A list of the top 10 files will be sent externally to Databricks with every request To avoid this, set the number of files to be sent to 0 using `chattr::chattr_defaults(max_data_files = 0)` ! A list of the top 10 data.frames currently in your R session will be sent externally to Databricks with every request diff --git a/tests/testthat/test-backend-databricks.R b/tests/testthat/test-backend-databricks.R index 990a906..9bb90ec 100644 --- a/tests/testthat/test-backend-databricks.R +++ b/tests/testthat/test-backend-databricks.R @@ -4,7 +4,7 @@ test_that("Submit method works", { return("test return") } ) - def <- test_simulate_model("databricks-meta-llama3-70b.yml") + def <- test_simulate_model("databricks-meta-llama-3-3-70b.yml") expect_equal( ch_submit(def, "test"), "test return" @@ -32,7 +32,7 @@ test_that("Completion function works", { x } ) - def <- test_simulate_model("databricks-meta-llama3-70b.yml") + def <- test_simulate_model("databricks-meta-llama-3-3-70b.yml") expect_null( ch_databricks_complete( prompt = "test", @@ -57,7 +57,7 @@ test_that("Error when status is not 200", { x } ) - def <- test_simulate_model("databricks-meta-llama3-70b.yml") + def <- test_simulate_model("databricks-meta-llama-3-3-70b.yml") expect_error( ch_databricks_complete( prompt = "test", @@ -84,7 +84,7 @@ test_that("Missing host returns error", { test_that("Init messages work", { - def <- test_simulate_model("databricks-meta-llama3-70b.yml") + def <- test_simulate_model("databricks-meta-llama-3-3-70b.yml") def$max_data_files <- 10 def$max_data_frames <- 10 expect_snapshot(app_init_message(def)) diff --git a/tests/testthat/test-chattr-use.R b/tests/testthat/test-chattr-use.R index 5ecbc8b..76b7e3e 100644 --- a/tests/testthat/test-chattr-use.R +++ b/tests/testthat/test-chattr-use.R @@ -31,7 +31,11 @@ test_that("Missing token prevents showing the option", { test_that("Menu works", { skip_on_cran() withr::with_envvar( - new = c("OPENAI_API_KEY" = "test", "DATABRICKS_TOKEN" = NA), + new = c( + "OPENAI_API_KEY" = "test", + "DATABRICKS_HOST" = NA, + "DATABRICKS_TOKEN" = NA + ), { local_mocked_bindings( menu = function(...) { diff --git a/vignettes/backend-databricks.Rmd b/vignettes/backend-databricks.Rmd index 9e55b3c..8f2cab0 100644 --- a/vignettes/backend-databricks.Rmd +++ b/vignettes/backend-databricks.Rmd @@ -22,17 +22,16 @@ knitr::opts_chunk$set( [Databricks](https://docs.databricks.com/en/introduction/index.html) customers have access to [foundation model APIs](https://docs.databricks.com/en/machine-learning/foundation-models/index.html) -like DBRX, Meta Llama 3 70B, and Mixtral 8x7B. Databricks also provides -the ability to train and [deploy custom +like Meta Llama 3.3 70B and 3.1 405B. Databricks also provides the +ability to train and [deploy custom models](https://docs.databricks.com/en/machine-learning/foundation-models/deploy-prov-throughput-foundation-model-apis.html). `chattr` supports the following models on Databricks by default: -| Model | Databricks Model Name | `chattr` Name | +| Model | Databricks Model Name | `chattr` Name | |---------------------|------------------------------|---------------------| -| DBRX Instruct | `databricks-dbrx-instruct` | `databricks-dbrx` | -| Meta-Llama-3-70B-Instruct | `databricks-meta-llama-3-70b-instruct` | `databricks-meta-llama3-70b` | -| Mixtral-8x7B Instruct | `databricks-mixtral-8x7b-instruct` | `databricks-mixtral8x7b` | +| Meta Llama 3.3 70B Instruct | `databricks-meta-llama-3-3-70b-instruct` | `databricks-meta-llama3.3-70b` | +| Meta Llama 3.1 405B Instruct | `databricks-meta-llama-3-1-405b-instruct` | `databricks-meta-3.1-405b` | : [Supported Databricks pay-per-token foundation models](https://docs.databricks.com/en/machine-learning/foundation-models/index.html#pay-per-token-foundation-model-apis) @@ -84,12 +83,12 @@ DATABRICKS_TOKEN = #################### ### Supported Models By default, `chattr` is setup to interact with GPT 4 (`gpt-4`). To -switch to Meta Llama 3 70B you can run: +switch to Meta Llama 3.3 70B you can run: ```{r} library(chattr) -chattr_use("databricks-meta-llama3-70b") +chattr_use("databricks-meta-llama-3-3-70b") ``` #### Custom Models @@ -107,9 +106,9 @@ endpoint name of `"CustomLLM"`: ```{r} library(chattr) -# use any existing databricks foundation model name (e.g. datarbicks-dbrx) -# then adjust the default model name to 'CustomMixtral' -chattr_use(x = "databricks-dbrx", model = "CustomLLM") +# use any existing databricks foundation model name (e.g. databricks-meta-3.1-405b) +# then adjust the default model name to 'CustomLLM' +chattr_use(x = "databricks-meta-llama-3-1-405b", model = "CustomLLM") ``` ## Data files and data frames @@ -139,4 +138,3 @@ time you start the Shiny app: ! A list of the top 10 files will be sent externally to Databricks with every request To avoid this, set the number of files to be sent to 0 using chattr::chattr_defaults(max_data_files = 0)Î ``` -