Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions inference-platforms/AgC/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# AgC - Agentic Compute

This shows how to use the AgC as an [OpenAI Responses adapter][docs],
using its [OpenTelemetry configuration][config].

AgC API requests are adapted and forwarded to Ollama as chat
completions.

## Prerequisites

Start Ollama and your OpenTelemetry Collector via this repository's [README](../README.md).

## Run AgC

```bash
docker compose up --pull always --force-recreate --remove-orphans
```

Clean up when finished, like this:

```bash
docker compose down
```

## Call AgC with python

Once AgC is running, use [uv][uv] to make an OpenAI request via
[chat.py](../chat.py):

```bash
# Set the OpenAI base URL to the AgC proxy, not Ollama
OPENAI_BASE_URL=http://localhost:6644/v1 uv run --exact -q --env-file env.local ../chat.py
```

Or, for the AgC Responses API
```bash
OPENAI_BASE_URL=http://localhost:6644/v1 uv run --exact -q --env-file env.local ../chat.py --use-responses-api
```

## Notes

AgC comes up with a platform service: open-responses (a Spring Boot application), so signals collected are adapted to
OpenTelemetry via a Otel-SDK.

---
[doc]: https://github.com/masaic-ai-platform/AgC
[config]: https://github.com/masaic-ai-platform/AgC/blob/main/platform/README.md#setting-up-the-opentelemetry-collector
[uv]: https://docs.astral.sh/uv/getting-started/installation/
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
services:
open-responses:
image: masaicai/open-responses:0.3.2
image: masaicai/open-responses:0.5.2
container_name: open-responses
env_file:
- env.local
ports:
- "8080:8080"
- "6644:6644"
extra_hosts: # send localhost traffic to the docker host, e.g. your laptop
- "localhost:host-gateway"
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_API_KEY=unused
CHAT_MODEL=qwen3:0.6B

# Disabled by default in open-responses
# Disabled by default in AgC
OTEL_SDK_DISABLED=false

OTEL_SERVICE_NAME=open-responses
Expand Down
4 changes: 2 additions & 2 deletions inference-platforms/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Elastic Stack.
* [Envoy AI Gateway](aigw) - with [OpenTelemetry tracing and metrics][aigw]
* [LiteLLM](litellm) - with [OpenTelemetry logging callbacks][litellm]
* [LlamaStack](llama-stack) - with [OpenTelemetry sinks][llama-stack]
* [OpenResponses](open-responses) - with [OpenTelemetry export][open-responses]
* [AgC](AgC) - with [OpenTelemetry export][AgC]
* [vLLM](vllm) - with [OpenTelemetry POC][vllm] configuration

If you use Elastic Stack, an example would look like this in Kibana:
Expand Down Expand Up @@ -109,7 +109,7 @@ To start and use Ollama, do the following:
[archgw]: https://docs.archgw.com/guides/observability/tracing.html
[litellm]: https://llama-stack.readthedocs.io/en/latest/building_applications/telemetry.html#configuration
[llama-stack]: https://llama-stack.readthedocs.io/en/latest/building_applications/telemetry.html#telemetry
[open-responses]: https://github.com/masaic-ai-platform/docs/blob/main/openresponses/observability.mdx
[AgC]: https://github.com/masaic-ai-platform/AgC/blob/main/platform/README.md#setting-up-the-opentelemetry-collector
[vllm]: https://github.com/vllm-project/vllm/blob/main/examples/online_serving/opentelemetry/README.md
[uv]: https://docs.astral.sh/uv/getting-started/installation/
[ollama-dl]: https://ollama.com/download
Expand Down
48 changes: 0 additions & 48 deletions inference-platforms/open-responses/README.md

This file was deleted.