Skip to content

[BUG]: aira-backend breaks when enabling phoenix telemetry #74

@yxchia98

Description

@yxchia98

Version

v1.2.0

Which installation method(s) does this occur on?

Docker

Describe the bug.

Following the guide to enable phoenix telemetry over at https://github.com/NVIDIA-AI-Blueprints/aiq-research-assistant/blob/v1.2.0/docs/phoenix-tracing.md, i encountered an error where aira-backend crashes when deploying using Docker Compose

general:
  use_uvloop: true

  telemetry:
    logging:
      console:
        _type: console
        level: WARN
    tracing:
      phoenix:
        _type: phoenix
        endpoint: http://phoenix:6006/v1/traces
        project: ai_researcher

Minimum reproducible example

Relevant log output

`2025-10-30 16:56:47,077 - nat.cli.commands.start - INFO - Starting NAT from config file: '/app/configs/config.yml'
2025-10-30 16:56:47,083 - nat.data_models.config - ERROR - Requested tracing type `phoenix` not found. Have you ensured the necessary package has been installed with `uv pip install`?
Available tracing names:
 - nat.plugins.opentelemetry/langfuse
 - nat.plugins.opentelemetry/langsmith
 - nat.plugins.opentelemetry/otelcollector
 - nat.plugins.opentelemetry/patronus
 - nat.plugins.opentelemetry/galileo
 - nat.observability/file
 - nat.plugins.weave/weave

2025-10-30 16:56:47,083 - nat.utils.exception_handlers.schemas - ERROR - Invalid configuration: general: Input tag 'phoenix' found using discriminator() does not match any of the expected tags: 'nat.plugins.opentelemetry/langfuse', 'nat.plugins.opentelemetry/langsmith', 'nat.plugins.opentelemetry/otelcollector', 'nat.plugins.opentelemetry/patronus', 'nat.plugins.opentelemetry/galileo', 'nat.observability/file', 'nat.plugins.weave/weave', 'langfuse', 'langsmith', 'otelcollector', 'patronus', 'galileo', 'file', 'weave'
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/nat/utils/exception_handlers/schemas.py", line 72, in inner_function
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/nat/utils/data_models/schema_validator.py", line 30, in validate_schema
    raise e
  File "/app/.venv/lib/python3.12/site-packages/nat/utils/data_models/schema_validator.py", line 27, in validate_schema
    return Schema(**metadata)
           ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/pydantic/main.py", line 214, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for Config
general.telemetry.tracing.phoenix
  Input tag 'phoenix' found using discriminator() does not match any of the expected tags: 'nat.plugins.opentelemetry/langfuse', 'nat.plugins.opentelemetry/langsmith', 'nat.plugins.opentelemetry/otelcollector', 'nat.plugins.opentelemetry/patronus', 'nat.plugins.opentelemetry/galileo', 'nat.observability/file', 'nat.plugins.weave/weave', 'langfuse', 'langsmith', 'otelcollector', 'patronus', 'galileo', 'file', 'weave' [type=union_tag_invalid, input_value={'_type': 'phoenix', 'end...oject': 'ai_researcher'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/union_tag_invalid

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/bin/nat", line 10, in <module>
    sys.exit(run_cli())
             ^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/nat/cli/main.py", line 40, in run_cli
    cli(obj={}, auto_envvar_prefix='NAT', show_default=True, prog_name="nat")
  File "/app/.venv/lib/python3.12/site-packages/click/core.py", line 1442, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/click/core.py", line 1363, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/click/core.py", line 1226, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/click/core.py", line 794, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/nat/cli/commands/start.py", line 188, in invoke_subcommand
    config = validate_schema(config_dict, Config)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/nat/utils/exception_handlers/schemas.py", line 77, in inner_function
    raise ValueError(log_error_message) from e
ValueError: Invalid configuration: general: Input tag 'phoenix' found using discriminator() does not match any of the expected tags: 'nat.plugins.opentelemetry/langfuse', 'nat.plugins.opentelemetry/langsmith', 'nat.plugins.opentelemetry/otelcollector', 'nat.plugins.opentelemetry/patronus', 'nat.plugins.opentelemetry/galileo', 'nat.observability/file', 'nat.plugins.weave/weave', 'langfuse', 'langsmith', 'otelcollector', 'patronus', 'galileo', 'file', 'weave'
ubuntu@ubuntu:~/aiq-research-assistant$ docker compose -f deploy/compose/docker-compose.yaml --profile aira down`

Full env printout

Other/Misc.

No response

Code of Conduct

  • I agree to follow THIS PROJECT's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report

Metadata

Metadata

Assignees

No one assigned

    Labels

    AIQ1.0Version 1.0bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions