AIServer

A local AI API server compatible with OpenAI, written in Python.

• Uses llama-cpp-python, stable-diffusion-cpp-python, pywhispercpp, piper-tts as backend for AI services.

Installation

1. Requirements:

• Python 3.11+ (3.11 is recommanded and tested)
• C compiler
  • Linux: gcc or clang
  • Windows: Visual Studio or MinGW
  • MacOS: Xcode
• ffmpeg library (Ubuntu Linux: sudo apt install ffmpeg)
• git

2. Clone this repository.

To clone this repository, run:

git clone https://github.com/smbhuin/aiserver.git
cd aiserver

3. Install python dependencies

First create virtual environment with python 3.11 (Recommanded) and activate.

python3 -m venv .venv
source .venv/bin/activate

To install the basic python dependencies, run:

pip install -r requirements.txt

To install individual backend packages, follow their own instructions.

• llama-cpp-python
• stable-diffusion-cpp-python
• pywhispercpp
• piper-tt

or to install pre-build cpu packages:

pip install -r requirements_cpu.txt

or to install gpu cuda packages:

pip install -r requirements_gpu.txt

To use transcribe api with files other than wav, you need to install ffmpeg:

sudo apt install ffmpeg

4. Download models

Download your prefered models supported by backends used in this project.

For example:

• LLM Models

  • Mistral 7B Instruct GGUF
  • Llama 3.1 8B Instruct GGUF

• SD/SDXL/Flux Models

  • Stable Diffusion 1.5

• Whisper Models

  • Whisper GGML

• Piper TTS Voices

  • Piper Voices

After downloading all the models models dir shall look like folowing:

models
│   qwen2.5-coder-7b-instruct-q5_k_m.gguf
│   mistral-7b-instruct-v0.2.Q4_K_M.gguf
|   nomic-embed-text-v1.5.f16.gguf
│
└───stable-diffusion
│   │   v1-5-pruned-emaonly.safetensors
│   │   dreamshaper_8.safetensors
│
└───audio
    │   en_US-amy-medium.onnx
    │   en_US-amy-medium.onnx.json
    |   whisper-small-ggml.bin
    |
    |

5. Create the config file

Rename aiserver.example.yml to aiserver.yml and edit to include your models.

6. Run the API server

python main.py

Usage

Open http://localhost:8000/docs for OpenAPI docs.

Chat Completion:

from openai import OpenAI

client = OpenAI(api_key="demo", base_url="http://localhost:8000/v1")

completion = client.chat.completions.create(
  model="mistral-7b",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

print(completion.choices[0].message)

Image Generation:

from openai import OpenAI

client = OpenAI(api_key="demo", base_url="http://localhost:8000/v1")

response = client.images.generate(
    model="dreamshaper-8",
    prompt="a white siamese cat",
    size="512x512",
    quality="standard",
    n=1,
)

print(response.data[0].url)

Speech To Text:

from openai import OpenAI

client = OpenAI(api_key="demo", base_url="http://localhost:8000/v1")
audio_file= open("/path/to/file/audio.mp3", "rb")
transcription = client.audio.transcriptions.create(
    model="whisper-1", 
    file=audio_file
)

print(transcription.text)

Text To Speech:

from openai import OpenAI

client = OpenAI(api_key="demo", base_url="http://localhost:8000/v1")
speech_file_path = "files/speech.mp3"
with client.audio.speech.with_streaming_response.create(
    model="tts-1",
    voice="amy",
    input="Today is a wonderful day to build something people love!",
) as response:
    response.stream_to_file(speech_file_path)

Discussions and contributions

If you find any bug, please open an issue.

If you have any feedback, or you want to share how you are using this project, feel free to use the Discussions and open a new topic.

Credits

• llama-cpp-python
• openedai-images

License

This project is licensed under MIT License.