Implement Tracing Integration

[LockedThread]

Overview

The new feature bridges Python logging with Rust’s tracing ecosystem, allowing for enhanced diagnostics and a unified logging interface across both languages. This addition is especially useful for projects that rely on structured logging and require detailed, contextual log output for debugging complex issues.

Key Changes

• Tracing Implementation:
  Integrated a new tracing backend that captures and forwards log events from Python into Rust’s tracing framework. This includes mapping Python log levels to their corresponding tracing levels for consistency. Additionally, I added support for KV pairs via Json serialization.

• Code Modifications:

  • Updated the logging initialization code to support the new tracing functionality.
  • Introduced conditional feature flags to enable or disable tracing support.
  • Adjusted existing logger configuration to ensure seamless integration without disrupting current functionality.

• Documentation & Examples:
  Updated README and inline documentation to cover the usage of the new tracing feature. Examples have been added to demonstrate how to configure and use the tracing integration effectively.

Example code

• A new example is included to show proper usage of tracing with KV support.

Disclaimer

• JSON-Based Serialization:
  The implementation relies on JSON-based serialization for log messages. This approach is necessary due to limitations in tracing, which currently does not support runtime-dynamic fields on log messages. For further details, see tokio-rs/tracing issue #372.

• Dependency on serde-pyobject:
  This PR uses the branch feat/loosen-deps from my serde-pyobject fork. We must wait until these changes are merged into the main branch to support this due to version constraints. See Loosen dependency version constraints for pyo3, serde, and serde_json Jij-Inc/serde-pyobject#21.
  Additionally, if Enhance Python Object Deserialization via __dict__ Field Jij-Inc/serde-pyobject#22 and Extend Deserialization Support to Pydantic Models and Dataclasses Jij-Inc/serde-pyobject#23 can be swiftly merged then we will be in really good standing. They add support for passing python classes, Pydantic BaseModels, and dataclasses directly into the logging KV pairs.

Impact and Future Work

• Backward Compatibility:
  Existing users of pyo3-pylogger will experience no changes unless they opt-in to the new tracing functionality. However, we had to increase the minimum support pyo3 version to 0.23 to include serde-pyobject. This must be at least a minor version update.

Maintain DRAFT status until Jij-Inc/serde-pyobject#21 is merged.

Your feedback is welcome, and I look forward to discussing any improvements or adjustments needed for this feature.

Please review and let me know if there are any questions or additional test cases that should be considered.

[dylanbstorey]

Looks ok to me, going to ask we hold off on the merge until upstream changes get merged in just to make sure dependencies are stable.

[dylanbstorey]

Also - go ahead and do a bump to 0.5 on your PR.

[LockedThread]

Looks ok to me, going to ask we hold off on the merge until upstream changes get merged in just to make sure dependencies are stable.

Agreed.

[dylanbstorey]

@LockedThread - i’m afraid that upstream PR isn’t going anywhere. Someone mentioned pythonize as a potential workaround for serde-pyobject. Have you looked at it ?

[LockedThread]

@LockedThread - i’m afraid that upstream PR isn’t going anywhere. Someone mentioned pythonize as a potential workaround for serde-pyobject. Have you looked at it ?

😞 Unfortunately yeah. I have been using that PR'd version in production for months without issue. I currently do not have the bandwidth to switch to pythonize because there is a large chunk of my company's code base using serde-pyobject by now.

[dylanbstorey]

alright - i’m going to merge with the git branch, its not my favorite thing but nbd

[dylanbstorey]

well poop - TIL that crates won’t accept a git i’ll see about using pythonize

[dylanbstorey]

I switched over to pythonize so it at least should work now.