Skip to content

Config

Jump to bottom
v2ray edited this page Jun 7, 2023 · 9 revisions

Overview

config.json is the config file, currently you can't modify the config inside the program, you have to modify the file directly.

Table of Contents

General Elements

You can modify these elements' values to change the config, but don't modify the elements' names.
Below elements' info follow this format and are placed in alphabetically ascending order:
element_name [element type 1 | element type 2]

api_base_url [string]

For settings the API base url, you can set this to another base url to use a custom proxy API.
Example:

"api_base_url": "https://api.openai.com"

api_key [string | array of string | null]

For setting the API key, it can be a string, an array of string to set multiple API keys in case one fails, or null if there's no API key saved.
Examples:

"api_key": "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxExampleKey1"
"api_key": [
  "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxExampleKey1",
  "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxExampleKey2"
]
"api_key": null

debug_reference [boolean]

For setting whether to show the constructed initial prompt or not, set it to true to see how this program handles embeddings internally.
Example:

"debug_reference": false

frequency_penalty [float]

For setting the frequency penalty. Click me for the explanation.
Example:

"frequency_penalty": 0.0

logit_bias [json object]

For setting the logit bias, can be used to promote or ban specific tokens. Click me for the explanation.
Example:
Hi, Hello and Hey are the texts to change the appear probability, 5.67, -5.67 and -100 are the bias values.
In this example, Hi has a higher chance to appear, Hello has a lower chance to appear, and Hey won't appear at all.
Bias values need to be >= -100 && <= 100.

"logit_bias": {
  "Hi": 5.67,
  "Hello": -5.67,
  "Hey": -100
}

max_display_length [unsigned integer]

For setting the maximum exchanges(prompt-response pair) the CLI will display.
Example:

"max_display_length": 100

max_reference_length [unsigned integer]

For setting the maximum exchanges(prompt-response pair) the program will use as the long-term memory(The embeddings searching result), the value is not dependent on max_short_memory_length.
Example:

"max_reference_length": 4

max_short_memory_length [unsigned integer]

For setting the maximum exchanges(prompt-response pair) the program will use as the short-term memory(The most recent exchanges), the value is not dependent on max_reference_length.
Example:

"max_short_memory_length": 4

max_tokens [integer]

For setting the maximum tokens the model can generate, if set to 500, then the model can only generate maximum of 500 tokens in one prompt.
Example:

"max_tokens": 500

model [string]

For setting which model the program will use, it can only use chat completion and text completion models, click here to see the model list.
Example:

"model": "gpt-3.5-turbo"

presence_penalty [float]

For setting the presence penalty. Click me for the explanation.
Example:

"presence_penalty": 0.6

search_response [boolean]

For setting whether to generate&search response embeddings as well or only search for input embeddings.
Generating response embeddings won't take additional time because I batched it up with the input embeddings generation request.
On the other hand, searching for both input and response embeddings will take double the time, and the saved chat history will be double the size too, but it won't be noticeable.
Example:

"search_response": true

space_between_exchanges [boolean]

For setting whether to add new lines between exchanges or not.
Example:

"space_between_exchanges": false
Me: hi
You: Hello! How may I assist you today?
Me: say 4 words
You: Sure! "Sunshine, adventure, happiness, love."
Me: _
"space_between_exchanges": true
Me: hi
You: Hello! How may I assist you today?

Me: say 4 words
You: Sure! "Sunshine, adventure, happiness, love."

Me: _

temperature [float]

For setting the temperature parameter for the model, higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
It's best to alter this or top_p but not both.
Example:

"temperature": 1.0

top_p [float]

For setting the top_p parameter for the model, it's an alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.
It's best to alter this or temperature but not both.
Example:

"top_p": 1.0

OS Specific Elements

Some elements will only show/work on specific operating systems, these elements will be explaned here.
Below elements' info follow this format and are placed in alphabetically ascending order:
element_name [element type 1 | element type 2] {OS 1 | OS 2}

ca_bundle_path [string] {Linux}

For setting the SSL CA certification bundle file in case the default path set by me is not correct in your system, you can modify this element to switch to a different ca-bundle.crt.
Example:

"ca_bundle_path": "/etc/ssl/certs/ca-bundle.crt"

Clone this wiki locally