Conversation
There was a problem hiding this comment.
Pull Request Overview
This pull request enhances the Microsoft Agents A365 observability library by adding new agent type support, caller details tracking, and extended telemetry methods. The changes improve the observability framework's ability to capture metadata about agents, their callers, and execution contexts.
- Introduces
AgentTypeenum andCallerDetailsmodel for better agent classification and caller tracking - Adds new telemetry recording methods to scopes (
record_input_messages,record_output_messages,record_response, etc.) - Updates copyright headers from "Microsoft. All rights reserved." to "Microsoft Corporation. Licensed under the MIT License."
- Adds comprehensive unit tests for the new telemetry methods
Reviewed Changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
models/agent_type.py |
Introduces AgentType enum with 5 agent classifications |
models/caller_details.py |
Adds CallerDetails dataclass for tracking caller information |
models/__init__.py |
Initializes models module with copyright header |
agent_details.py |
Expands AgentDetails with new fields (auid, upn, blueprint_id, agent_type, tenant_id) and updates copyright |
constants.py |
Adds new constants for agent type and thought process tracking |
opentelemetry_scope.py |
Sets additional agent detail tags in telemetry spans |
invoke_agent_scope.py |
Adds caller tracking, execution metadata, and message recording methods; updates copyright |
inference_scope.py |
Adds methods for recording messages, tokens, response IDs, finish reasons, and thought processes; updates copyright |
execute_tool_scope.py |
Adds record_response method and refactors variable extraction; updates copyright |
trace_processor/util.py |
Includes GEN_AI_AGENT_TYPE_KEY in baggage attributes list |
test_invoke_agent_scope.py |
Unit tests verifying existence of new recording methods on InvokeAgentScope |
test_inference_scope.py |
Unit tests for new InferenceScope recording methods |
test_execute_tool_scope.py |
Unit test for ExecuteToolScope record_response method |
Comments suppressed due to low confidence (2)
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/constants.py:21
- The constant name contains the keyword 'Kairo' which should be removed or replaced with appropriate terminology. Consider renaming to
ENABLE_LEGACY_EXPORTERor removing if no longer needed, given that the codebase is transitioning away from 'Kairo' references.
ENABLE_KAIRO_EXPORTER = "ENABLE_KAIRO_EXPORTER"
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/constants.py:107
- The constant name contains the keyword 'Kairo' which should be removed or replaced. While this is marked as 'Legacy constant for backward compatibility', consider deprecation strategy or renaming to
LEGACY_AGENT_ID_KEY.
KAIRO_AGENT_ID_KEY = AGENT_ID_KEY
...gents-a365-observability-core/microsoft_agents_a365/observability/core/invoke_agent_scope.py
Outdated
Show resolved
Hide resolved
...gents-a365-observability-core/microsoft_agents_a365/observability/core/invoke_agent_scope.py
Outdated
Show resolved
Hide resolved
...t-agents-a365-observability-core/microsoft_agents_a365/observability/core/inference_scope.py
Outdated
Show resolved
Hide resolved
...t-agents-a365-observability-core/microsoft_agents_a365/observability/core/inference_scope.py
Outdated
Show resolved
Hide resolved
...t-agents-a365-observability-core/microsoft_agents_a365/observability/core/inference_scope.py
Outdated
Show resolved
Hide resolved
|
@nikhilNava I've opened a new pull request, #19, to work on those changes. Once the pull request is ready, I'll request review from you. |
… asserts, and more - Fix #1: Downgrade sensitive data logging from INFO to DEBUG in agent365_exporter.py - Fix #2: Fix unpaired context.attach() in opentelemetry_scope.py add_baggage() by storing and detaching baggage tokens on scope end - Fix #3: Add bounded OrderedDict caps to unbounded dicts in OpenAI trace_processor.py - Fix #4: Replace 30 assert statements with proper TypeError raises in LangChain utils.py - Fix #5: Log security warning when HTTP domain override is detected - Fix #6: Warn when bearer token sent over non-HTTPS connection - Fix #10: Respect Retry-After header and use exponential backoff in retries - Fix #13: Rename reset() to _reset() in ObservabilityHostingManager - Fix #15: Replace print() with logger.warning() in LangChain tracer_instrumentor.py Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com>
* Initial plan * Replace assert statements with explicit TypeError raises in langchain utils Replace all 30 assert statements in utils.py with equivalent if-not-raise TypeError checks. This ensures type validation is not silently stripped when Python runs with -O (optimized mode). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * security: fix sensitive data logging, context leak, unbounded memory, asserts, and more - Fix #1: Downgrade sensitive data logging from INFO to DEBUG in agent365_exporter.py - Fix #2: Fix unpaired context.attach() in opentelemetry_scope.py add_baggage() by storing and detaching baggage tokens on scope end - Fix #3: Add bounded OrderedDict caps to unbounded dicts in OpenAI trace_processor.py - Fix #4: Replace 30 assert statements with proper TypeError raises in LangChain utils.py - Fix #5: Log security warning when HTTP domain override is detected - Fix #6: Warn when bearer token sent over non-HTTPS connection - Fix #10: Respect Retry-After header and use exponential backoff in retries - Fix #13: Rename reset() to _reset() in ObservabilityHostingManager - Fix #15: Replace print() with logger.warning() in LangChain tracer_instrumentor.py Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Restore agent/tenant IDs and response text in exporter log messages Agent IDs and tenant IDs are not sensitive data and are useful for debugging. Restore them in debug/error log messages. Also restore truncated response text in HTTP error logs to help developers debug failures. Log levels remain at DEBUG (from the prior security fix). Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Remove add_baggage() from OpenTelemetryScope The method had an unpaired context.attach() that leaked context tokens. Users should use BaggageBuilder.build() context manager instead, which properly restores the previous context on exit. Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Security hardening for observability packages Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Fix formatting in test_agent365_exporter.py and replace remaining raise TypeError with isinstance guards in langchain utils.py Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Move _parse_retry_after to exporters/utils.py as standalone parse_retry_after function Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Replace type(e).__name__ with str(e) in exporter error logging per PR review Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * feat: add bounded collections for LangChain tracer and OutputScope - Convert LangChain _spans_by_run from unbounded DictWithLock to bounded OrderedDict with _MAX_TRACKED_RUNS=10000 cap - Add _cap_ordered_dict helper for FIFO eviction (matching OpenAI pattern) - Add thread-safe lock usage for _spans_by_run in error handlers - Add _MAX_OUTPUT_MESSAGES=5000 cap for OutputScope._output_messages - Add unit tests for both bounded collections Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> Co-authored-by: Nikhil Navakiran <nikhil.navakiran@gmail.com>
* Initial plan * Replace assert statements with explicit TypeError raises in langchain utils Replace all 30 assert statements in utils.py with equivalent if-not-raise TypeError checks. This ensures type validation is not silently stripped when Python runs with -O (optimized mode). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * security: fix sensitive data logging, context leak, unbounded memory, asserts, and more - Fix microsoft#1: Downgrade sensitive data logging from INFO to DEBUG in agent365_exporter.py - Fix microsoft#2: Fix unpaired context.attach() in opentelemetry_scope.py add_baggage() by storing and detaching baggage tokens on scope end - Fix microsoft#3: Add bounded OrderedDict caps to unbounded dicts in OpenAI trace_processor.py - Fix microsoft#4: Replace 30 assert statements with proper TypeError raises in LangChain utils.py - Fix microsoft#5: Log security warning when HTTP domain override is detected - Fix microsoft#6: Warn when bearer token sent over non-HTTPS connection - Fix microsoft#10: Respect Retry-After header and use exponential backoff in retries - Fix microsoft#13: Rename reset() to _reset() in ObservabilityHostingManager - Fix microsoft#15: Replace print() with logger.warning() in LangChain tracer_instrumentor.py Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Restore agent/tenant IDs and response text in exporter log messages Agent IDs and tenant IDs are not sensitive data and are useful for debugging. Restore them in debug/error log messages. Also restore truncated response text in HTTP error logs to help developers debug failures. Log levels remain at DEBUG (from the prior security fix). Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Remove add_baggage() from OpenTelemetryScope The method had an unpaired context.attach() that leaked context tokens. Users should use BaggageBuilder.build() context manager instead, which properly restores the previous context on exit. Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Security hardening for observability packages Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Fix formatting in test_agent365_exporter.py and replace remaining raise TypeError with isinstance guards in langchain utils.py Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Move _parse_retry_after to exporters/utils.py as standalone parse_retry_after function Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * Replace type(e).__name__ with str(e) in exporter error logging per PR review Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> * feat: add bounded collections for LangChain tracer and OutputScope - Convert LangChain _spans_by_run from unbounded DictWithLock to bounded OrderedDict with _MAX_TRACKED_RUNS=10000 cap - Add _cap_ordered_dict helper for FIFO eviction (matching OpenAI pattern) - Add thread-safe lock usage for _spans_by_run in error handlers - Add _MAX_OUTPUT_MESSAGES=5000 cap for OutputScope._output_messages - Add unit tests for both bounded collections Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: nikhilNava <211831449+nikhilNava@users.noreply.github.com> Co-authored-by: Nikhil Navakiran <nikhil.navakiran@gmail.com>
Task
Introduce agent type and other helper attributes and methods on the SDK