PyCarol

PyCarol is a Python SDK designed to support data ingestion and data access workflows on Carol. It provides abstractions for authentication, connector and staging management, data ingestion, and querying, enabling reliable integration with Carol services using Python. The SDK encapsulates low-level API communication and authentication logic, making data pipelines easier to build, maintain, and operate.

Table of Contents

• Getting Started
• Recommended authentication method
• Explicit authentication methods
  • Using user/password
  • Using Tokens
  • Using API Key
• Setting up Carol entities
• Sending Data
• Reading data
• Carol In Memory
• Logging
  • Prerequisites
  • Logging messages to Carol
  • Notes
• Calling Carol APIs
• Settings
• Useful Functions
• Release process

Getting Started

Run pip install pycarol to install the latest stable version from PyPI. Documentation is hosted on Read the Docs.

Recommended authentication method

Never write passwords or API tokens in plain text. Use environment variables whenever possible.

Carol URL format:

www.ORGANIZATION.carol.ai/TENANT_NAME

Explicit authentication methods

Carol is the main object to access pyCarol and Carol APIs.

Using user/password

from pycarol import PwdAuth, Carol

carol = Carol(
    domain=TENANT_NAME,
    app_name=APP_NAME,
    auth=PwdAuth(USERNAME, PASSWORD),
    organization=ORGANIZATION
)

Using Tokens

from pycarol import PwdKeyAuth, Carol

carol = Carol(
    domain=TENANT_NAME,
    app_name=APP_NAME,
    auth=PwdKeyAuth(pwd_auth_token),
    organization=ORGANIZATION
)

Using API Key

from pycarol import ApiKeyAuth, Carol

carol = Carol(
    domain=DOMAIN,
    app_name=APP_NAME,
    auth=ApiKeyAuth(api_key=X_AUTH_KEY),
    connector_id=CONNECTORID,
    organization=ORGANIZATION
)

Setting up Carol entities

from pycarol import Connectors

connector_id = Connectors(carol).create(
    name="my_connector",
    label="connector_label"
)

Sending Data

from pycarol import Staging

Staging(carol).send_data(
    staging_name="my_stag",
    data=[{"name": "Rafael"}],
    connector_id=CONNECTORID
)

Reading data

from pycarol import BQ, Carol

BQ(Carol()).query("SELECT * FROM stg_connectorname_tablename")

Carol In Memory

from pycarol import Memory

memory = Memory()
memory.add("my_table", [{"id": 1}])
memory.query("SELECT * FROM my_table")

Logging

Prerequisites

Set LONGTASKID when running locally.

Logging messages to Carol

import logging
from pycarol import CarolHandler, Carol

logger = logging.getLogger(__name__)
logger.addHandler(CarolHandler(Carol()))
logger.info("Hello Carol")

Notes

• Logs are linked to long tasks
• Console fallback when task ID is missing

Calling Carol APIs

carol.call_api("v1/some/endpoint", method="POST")

Settings

from pycarol.apps import Apps
Apps(carol).get_settings(app_name="my_app")

Useful Functions

from pycarol.functions import track_tasks
track_tasks(carol, ["task1", "task2"])

Release process

1. Open PR to main
2. Merge after approval
3. Update README if needed

Made with ❤ at TOTVS IDeIA