coeai - Professional Python Client for COE AI LLM API

Professional Python client for interacting with high-capacity multimodal Large Language Models hosted on the COE AI GPU cluster.

⚠️ Network Requirement: This API is only accessible from UPES's internal network (UPESNET). Ensure you are connected to UPES Wi-Fi to use this package.

The coeai package provides seamless LLM inference over the UPES local network, supporting both text-to-text and image-to-text operations with advanced streaming capabilities.

🚀 Quick Start

pip install coeai

from coeai import LLMinfer

# Initialize client (requires UPESNET connectivity)
llm = LLMinfer(api_key="your-api-key")

# Generate text
response = llm.generate(
    model="tinyllama:latest",
    prompt="Explain quantum computing in simple terms",
    max_tokens=256
)
print(response)

✨ Features

Feature	Description
Text Generation	Support for all available LLM models
Vision Models	Image-to-text with multimodal models
Streaming	Real-time response streaming
Error Handling	Comprehensive error messages with actionable guidance
File Management	Automatic cleanup of file handles
Logging	Optional debug logging for troubleshooting
Model Discovery	Programmatic model listing

📋 Available Models

Get the latest list programmatically:

models = llm.list_models()
print(f"Available models: {', '.join(models)}")

Current models (as of February 2026):

Text Models

tinyllama:latest - Compact model for basic tasks
tinyllama:1.1b - Small efficient model
deepseek-r1:70b - Advanced reasoning model
gpt-oss:120b - Large general-purpose model
llama4:16x17b - High-quality multimodal model
llama4:128x17b - Largest available model

Vision Models (Image + Text)

llama3.2-vision:11b - Vision-capable model
llama4:16x17b - Recommended for image analysis

Embedding Model

bge-m3:567m - Text embeddings model

📖 Usage Examples

1. Basic Text Generation

from coeai import LLMinfer

llm = LLMinfer(api_key="your-api-key")

response = llm.generate(
    model="tinyllama:latest",
    prompt="Write a haiku about programming",
    max_tokens=100,
    temperature=0.8
)
print(response)

2. Streaming Response

response = llm.generate(
    model="deepseek-r1:70b",
    prompt="Explain the theory of relativity",
    max_tokens=512,
    stream=True,
    print_stream=True  # Print as it generates
)

3. Single Image Analysis

response = llm.generate(
    model="llama3.2-vision:11b",
    inference_type="image-to-text",
    files=["/path/to/image.jpg"],
    prompt="Describe this image in detail",
    max_tokens=512
)
print(response)

4. Multiple Image Comparison

response = llm.generate(
    model="llama4:16x17b",
    inference_type="image-to-text",
    files=["/path/to/image1.jpg", "/path/to/image2.jpg"],
    prompt="Compare these images and identify differences",
    max_tokens=1024,
    temperature=0.7
)

5. Custom Conversation

messages = [
    {"role": "system", "content": [{"type": "text", "text": "You are a helpful coding assistant."}]},
    {"role": "user", "content": [{"type": "text", "text": "Explain Python decorators"}]}
]

response = llm.generate(
    model="gpt-oss:120b",
    messages=messages,
    max_tokens=512
)

6. Advanced Parameters

response = llm.generate(
    model="deepseek-r1:70b",
    prompt="Solve:  What is 15! (factorial)?",
    max_tokens=400,
    context_window=4096, # Increase context for complex tasks
    temperature=0.1,   # Lower = more deterministic
    top_p=0.9          # Nucleus sampling
)

🔧 API Reference

LLMinfer Class

Initialization

LLMinfer(api_key: str, host: str = "http://10.9.6.165:8000")

Parameter	Type	Description	Default
`api_key`	str	Your COE AI API key (required)	-
`host`	str	API server URL	`http://10.9.6.165:8000`

Methods

`generate()`

generate(
    model: str,
    inference_type: str = "text-to-text",
    prompt: Optional[str] = None,
    messages: Optional[List[Dict]] = None,
    files: Optional[List[str]] = None,
    max_tokens: int = 512,
    temperature: float = 0.7,
    top_p: float = 1.0,
    context_window: int = 2048,
    stream: bool = False,
    print_stream: bool = True
) -> Dict

Parameter	Type	Description
`model`	str	Model name (required)
`inference_type`	str	`"text-to-text"` or `"image-to-text"`
`prompt`	str	Text prompt (optional if `messages` provided)
`messages`	list	Custom conversation history
`files`	list	Image file paths for vision models
`max_tokens`	int	Maximum tokens to generate
`temperature`	float	Sampling temperature (0.0–2.0)
`top_p`	float	Nucleus sampling (0.0–1.0)
`context_window`	int	Context size (num_ctx), default 2048
`stream`	bool	Enable streaming response
`print_stream`	bool	Print stream to console

Returns: Dictionary with API response

`list_models()`

list_models() -> List[str]

Returns list of all available model names.

🔑 Getting an API Key

Option 1: Web Dashboard (Recommended)

Connect to UPESNET (UPES Wi-Fi)
Visit https://coeai.ddn.upes.ac.in
Sign in with your UPES credentials
Generate an API key from the dashboard

Option 2: Email Request

Send an email to hpc-access@ddn.upes.ac.in from your UPES account:

Subject: API Key Request for COE AI LLM Access

Dear COE AI Team,

I am requesting access to the LLM API for my project work.

Project Details:
- Project Name: <Your Project>
- Description: <Brief description>
- Expected Usage: <How you'll use the API>

Name: <Your Name>
Email: <Your UPES Email>
Department: <Your Department>

Thank you!

Allow 2-3 business days for processing.

⚠️ Error Handling

The client provides detailed error messages:

from coeai import LLMinfer, AuthenticationError, ModelNotFoundError, InferenceError

llm = LLMinfer(api_key="your-key")

try:
    response = llm.generate(
        model="tinyllama:latest",
        prompt="Hello world"
    )
except AuthenticationError as e:
    print(f"Auth failed: {e}")
except ModelNotFoundError as e:
    print(f"Model not found: {e}")
except InferenceError as e:
    print(f"Inference failed: {e}")
except FileNotFoundError as e:
    print(f"Image file missing: {e}")

Common Errors

Error	Cause	Solution
`AuthenticationError`	Invalid API key	Check key or get new one
`ModelNotFoundError`	Model doesn't exist	Use `llm.list_models()` to see available models
`InferenceError` (429)	Rate limit exceeded	Wait and retry
`FileNotFoundError`	Image path wrong	Verify file exists
`ConnectionError`	Can't reach server	Verify you're on UPESNET, check server status

🔍 Troubleshooting

Enable Debug Logging

import logging
logging.basicConfig(level=logging.DEBUG)

from coeai import LLMinfer
llm = LLMinfer(api_key="your-key")

Check Package Version

import coeai
print(f"coeai version: {coeai.__version__}")

Test Connection

llm = LLMinfer(api_key="your-key")
try:
    models = llm.list_models()
    print(f"✅ Connected! Found {len(models)} models")
except Exception as e:
    print(f"❌ Connection failed: {e}")

📊 Performance Tips

Model Selection
- Use tinyllama for quick responses
- Use deepseek-r1:70b for reasoning tasks
- Use llama4:16x17b or llama3.2-vision for image analysis
Temperature Settings
- 0.1-0.3: Factual/technical content
- 0.7-0.9: Balanced creativity
- 1.0-2.0: Maximum creativity
Token Limits
- Set max_tokens appropriately to balance quality and speed
- Typical: 100-256 for summaries, 512-1024 for detailed responses
Streaming
- Enable stream=True for long responses to see progress

🔄 Migration from v2.x

Breaking Changes in v4.0.0

⚠️ Important: Requires UPESNET (UPES internal network) connectivity

Update Your Code:

Before (v2.x/v3.x):

llm = LLMinfer(api_key="key", host="http://10.9.6.165:8001")  # Wrong port

After (v4.0.0):

# Correct - uses port 8000 (default)
llm = LLMinfer(api_key="key")

# Or explicitly specify
llm = LLMinfer(api_key="key", host="http://10.9.6.165:8000")

New Features in v4.0

✅ Automatic file handle cleanup
✅ Comprehensive error handling with custom exceptions
✅ list_models() method for model discovery
✅ Debug logging support
✅ Relaxed vision model restrictions
✅ Version export (coeai.__version__)

📜 Changelog

v4.0.0 (Current)

BREAKING: Fixed default port from 8001 to 8000
REQUIREMENT: Only accessible from UPES internal network (UPESNET)
API Keys: Get your key from https://coeai.ddn.upes.ac.in
Added list_models() method
Improved error handling with custom exceptions
Fixed file handle leaks
Added logging support
Relaxed vision model restrictions
Added __version__ export
Updated documentation with current models

v2.3.0

Production release with text-to-text and image-to-text support
Streaming capabilities
Multiple image processing

📄 License

Released under the MIT License. See LICENSE for details.

👥 Authors

Konal Puri & Sawai Pratap Khatri
Centre of Excellence: AI (COE AI), HPC Project, UPES

PyPI: https://pypi.org/project/coeai
GitHub: https://github.com/pkonal23/COE-AI-HPC-Project
API Server: http://10.9.6.165:8000 (UPESNET only)
Get API Key: https://coeai.ddn.upes.ac.in

🤝 Contributing

Contributions are welcome! Please feel free to submit issues or pull requests on GitHub.

📞 Support

For issues or questions:

Open an issue on GitHub
Contact: hpc-access@ddn.upes.ac.in

Made with ❤️ at UPES Centre of Excellence: AI

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
coeai-env		coeai-env
coeai.egg-info		coeai.egg-info
coeai		coeai
dist		dist
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
cURL_Instructions.md		cURL_Instructions.md
publish_package.sh		publish_package.sh
setup.py		setup.py
test_all_params.py		test_all_params.py
test_custom_messages.py		test_custom_messages.py
test_image_to_text.py		test_image_to_text.py
test_image_to_text_multiple.py		test_image_to_text_multiple.py
test_streaming_text.py		test_streaming_text.py
test_text_prompt.py		test_text_prompt.py

License

pkonal23/COE-AI-HPC-Project

Folders and files

Latest commit

History

Repository files navigation