Skip to content

MamtaTibco/New_OptInUser_code

BranchesTags

Repository files navigation

Transfer Carton Close Producer Service

A robust microservice for producing and publishing Transfer Carton Close messages, transforming and validating files, and integrating with downstream systems.

Overview

The service receives transfer carton close requests via REST API, validates and transforms the payload (XML/JSON), interacts with databases and external APIs, and publishes messages to Kafka and printer downstream systems.

Features

  • REST API Producer: Accepts transfer carton close requests via HTTP endpoints.
  • Payload Validation: Validates incoming requests using Pydantic models and XSD schemas (for XML payloads).
  • Transformation: Transforms data using XSLT and custom logic to required formats (XML, JSON, CSV).
  • Database Integration: Reads/writes to a backend database for transfer carton and related data.
  • Kafka/Message Publishing: Publishes processed messages to Kafka and printers for downstream consumption.
  • Health, Metrics, and Version Endpoints: Exposes /health, /metrics, and /version endpoints for observability and monitoring.
  • Structured Logging: All operations are logged in a structured format for easy troubleshooting and monitoring.
  • Error Handling: Graceful error handling with clear error messages and status codes.
  • Cloud-Native Ready: Designed for deployment on GCP, with best practices for containerization and scalability.

Architecture

The Transfer Carton Close Producer Service follows a layered, event-driven architecture that coordinates API input, validation, transformation, database access, external service integration, printing, and message publishing. Below is a detailed flow:

  1. API Layer

    • The main entry point is a PUT API endpoint: /transfers/transfer/{cartonNumber}.
      • Accepts the carton number as a path parameter and printer IP in the request body.
      • Validates input, processes the transfer carton close, and returns tracking and print status.
    • Receives transfer carton close requests via REST API (carton number, printer IP).
    • Passes request data to the service layer for processing.

  2. Service Layer (TransferCartonCloseService)

    • Orchestrates the end-to-end transfer carton close process:
      • Fetch Data: Retrieves transfer header and details from the database using repository classes.
      • Validation: Ensures there are items and non-zero quantity; updates header and returns early if not.
      • Store Lookup: Calls the StoreMisl web service for both ShipFrom and ShipTo stores to get store information.
      • Transformation: Transforms transfer data and store info into XML using XSLT and custom logic.
      • Service Lookup: Determines the shipping/printing service (e.g., GENERIC or Proship) using a lookup table.
      • Branching Logic:
        • If service is GENERIC: updates header and sends data to a generic printer.
        • Otherwise: processes data through the Proship service for advanced shipping/printing.
      • Kafka Messaging: Extracts ASN flag from a lookup table and sends the final XML message to Kafka for downstream processing.
      • Response: Returns a response with tracking number and print status.

  3. Integration Layer

    • Database: Uses repository classes for all database operations (fetching and updating transfer headers/details).
    • External APIs: Integrates with StoreMisl and Proship APIs for store and shipping information.
    • Printing: Supports both generic and Proship-based printing flows.
    • Kafka: Publishes messages to Kafka for further processing by other services.

  4. Error Handling

    • Handles and logs exceptions for transformation, database, and printing errors.
    • Returns appropriate HTTP error responses for each failure scenario.

  5. Observability

    • All operations are logged with structured logging.
    • Health, metrics, and version endpoints are exposed for monitoring.

Sequence Diagram:

Client/API Request
    |
    v
[FastAPI Route] ---> [TransferCartonCloseService]
                        |
                        |-- fetch transfer header/details (DB)
                        |-- validate items/quantity
                        |-- store lookup (StoreMisl API)
                        |-- transform CloseTransfer to XML (XSLT/custom)
                        |-- lookup proship service (table)
                        |-- branch: GENERIC -> print_generic()
                        |-- branch: Proship -> retrieve_proship -> process_proship()
                        |-- extract ASN flag (lookup)
                        |-- send to Kafka (MessageSender)
                        |
                        v
                  [Response to API]

Project Structure

  • src/ - Application source code
    • app.py - FastAPI app factory and server configuration
    • routes/ - API and health check routes
      • health/ - Health, metrics, and version endpoints
      • api/v1/ - Main business endpoints (e.g., /transfer-carton-close)
    • services/ - Business logic for processing, transformation, and integration
    • repositories/ - Database access and query logic
    • model/ - Data models (Pydantic/dataclasses) for requests and responses
    • utils/ - Utility functions (e.g., HTTP, XML, logging)
    • exception/ - Custom exception classes
    • settings/ - Application configuration
    • files/ - XSD, XSL, and YAML files for schema and lookup
  • tests/ - Unit and integration tests
    • unit/ - Unit tests for services, settings, and utils
    • integration/ - Integration tests for API routes
  • scripts/ - Utility scripts for authentication and connectivity tests
  • install.sh - Shell script to set up the environment
  • requirements.txt - Python dependencies
  • tox.ini / requirements-tox.txt - Tox configuration for testing

Requirements

Environment Variables

Set the following environment variables to configure the service:

Variable Name Description Default
UVICORN_PORT Port for the Uvicorn server 8080
UVICORN_WORKERS Number of Uvicorn worker processes 1
UVICORN_ACCESS_LOG Enable access logging for Uvicorn (True/False) True
UVICORN_RELOAD Enable hot reload for development (True/False) False
FASTAPI_ENABLE_DOCS Enable FastAPI Swagger/OpenAPI docs (True/False) False
X35_LOGGING_SERVICE_NAME Service name for structured logging
X35_LOGGING_LOG_LEVEL Log level for application logging (e.g., INFO, DEBUG)
APP_CONFIG Path to application YAML config file
ONSIE_DATABASE_HOST Hostname for the ONSIE database sushqdsqd03
ONSIE_DATABASE_PORT Port for the ONSIE database 1433
ONSIE_DATABASE_USER_NAME Username for the ONSIE database
ONSIE_DATABASE_PASSWORD Password for the ONSIE database
ONSIE_DATABASE_NAME Name of the ONSIE database OnsieTransferDev
ONSIE_DATABASE_DRIVER ODBC driver for the ONSIE database ODBC Driver 17 for SQL Server
ONSIE_DATABASE_ECHO Enable SQL query logging (True/False) False
ONSIE_DATABASE_POOL_SIZE Database connection pool size 10
ONSIE_DATABASE_MAX_OVERFLOW Temporary overload capacity for DB pool 20
ONSIE_DATABASE_POOL_TIMEOUT Timeout waiting for a DB connection (seconds) 30
ONSIE_DATABASE_POOL_RECYCLE Reconnect stale DB connections (seconds) 1800
ONSIE_DATABASE_POOL_PRE_PING Check if DB connection is still valid (True/False) True
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_BOOTSTRAP_SERVERS Kafka bootstrap servers
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_API_KEY Kafka SASL API key (username)
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_API_SECRET Kafka SASL API secret (password)
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_PRODUCER_ID Unique identifier for this Kafka producer transfer_carton_close_producer
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_TOPIC Kafka topic to produce transfer carton close messages to UO.CLOSE.TRANSFER.PUBLISH.TOPIC
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_ENABLE_IDEMPOTENCE Enable idempotent Kafka producer (True/False) True
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_DELIVERY_TIMEOUT_MS Kafka delivery timeout in milliseconds 120000
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION Max in-flight Kafka requests per connection 5
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_AUTO_REGISTER_SCHEMAS Auto-register Avro schemas with schema registry (True/False) False
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_USE_LATEST_VERSION Use latest schema version for Avro (True/False) True
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_CLIENT_DNS_LOOKUP Kafka client DNS lookup mode (e.g., use_all_dns_ips) use_all_dns_ips
KAFKA_TRANSFER_CARTON_CLOSE_PRODUCER_VALUE_SERIALIZER_TYPE Kafka value serializer type (str/avro) str
TRANSFER_CARTON_CLOSE_STOREMISL_URI StoreMisl API endpoint for store lookup http://d1shmisl01.phl.ecomm.urbanout.com/admin/api/v1/stores/search?showUnpublished=true&storeNumber=
TRANSFER_CARTON_CLOSE_PROSHIP_URI Proship API endpoint for shipping/printing http://tibcostage.urbanout.com:8545/proshipRequest/ship-print/v1
TRANSFER_CARTON_CLOSE_STOREMISL_TIMEOUT Timeout for StoreMisl API requests (ms) 10000
TRANSFER_CARTON_CLOSE_PROSHIP_TIMEOUT Timeout for Proship API requests (ms) 10000
TRANSFER_CARTON_CLOSE_PRINTER_PORT TCP port for printer communication 9100
TRANSFER_CARTON_CLOSE_PRINTER_CONNECTION_TIMEOUT TCP connection timeout for printer (seconds) 5
TRANSFER_CARTON_CLOSE_HTTPX_LOG_LEVEL Httpx Log level (e.g., DEBUG, INFO, WARNING, ERROR) WARNING
TRANSFER_CARTON_CLOSE_DICTTOXML_LOG_LEVEL dicttoxml Log level (e.g., DEBUG, INFO, WARNING, ERROR) WARNING

Usage

Setup

  1. Install

    # Clone your new repository
git clone <your-new-repo>
cd <your-new-repo>

./install.sh  # Install dependencies

source venv/bin/activate  # Activate virtual environment

  2. Run Tests

    # Run tests
tox

  3. Run Service w/Hot Reload

    PYTHONPATH=$(pwd) UVICORN_RELOAD=true python src/app.py

API Endpoints

Endpoint Method Description
/health GET Health check
/metrics GET Prometheus metrics
/version GET Service version info
/api/v1/transfer-carton-close POST Submit transfer carton close request

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages