Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
a1259b6
Modified SDK client to use backend endpoints
michal-chrobok Jun 18, 2025
313cdb8
Removed debug print
michal-chrobok Jun 18, 2025
94d7e0f
Removed pandas dependency
michal-chrobok Jun 18, 2025
4c7f47d
Updated pyarrow to 19.0.1
michal-chrobok Jun 18, 2025
87a6cec
Split examples into multiple scripts
michal-chrobok Jun 23, 2025
95ddad9
Fixed unit tests
michal-chrobok Jun 23, 2025
9f2db8b
Fixed lint issues
michal-chrobok Jun 23, 2025
53d2f3c
Ruff format
michal-chrobok Jun 23, 2025
33eeb57
mypy fixes
michal-chrobok Jun 23, 2025
7962fdd
Another ruff fix
michal-chrobok Jun 23, 2025
f2956e3
Added authentication to SDK. Modified examples.
michal-chrobok Jul 1, 2025
7625e6e
Small changes in example scritps
michal-chrobok Jul 1, 2025
9a9c696
Fixed tests and linter issues
michal-chrobok Jul 1, 2025
0916a33
Removed publishable key from examples
michal-chrobok Jul 2, 2025
ecf93c0
Merge branch 'feature/new-backend-endpoints' into fix/cvec-class-params
michal-chrobok Jul 2, 2025
9930c7e
Removed publishable key and email from cvec class params. Added api c…
michal-chrobok Jul 2, 2025
3000478
Fixed linting and tests.
michal-chrobok Jul 2, 2025
7f6c06a
Another lint fix
michal-chrobok Jul 2, 2025
a174451
fix: remove mention of CVEC_TENANT
joshuanapoli Jul 15, 2025
116204e
feat: integration example with all enpoints
baftinho Jul 29, 2025
1571ece
fix: add import
baftinho Jul 29, 2025
9679de9
fix: formatting
baftinho Jul 29, 2025
121774b
docs: Document `add_metric_data` function in README
joshuanapoli Jul 29, 2025
1b3e6a9
docs: Add example for `add_metric_data` to README
joshuanapoli Jul 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -173,3 +173,5 @@ cython_debug/
# PyPI configuration file
.pypirc
.aider*

.vscode/
41 changes: 38 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import cvec
from datetime import datetime
```

Construct the CVec client. The host, tenant, and api_key can be given through parameters to the constructor or from the environment variables CVEC_HOST, CVEC_TENANT, and CVEC_API_KEY:
Construct the CVec client. The host, tenant, and api_key can be given through parameters to the constructor or from the environment variables CVEC_HOST, and CVEC_API_KEY:

```
cvec = cvec.CVec()
Expand All @@ -48,7 +48,7 @@ The newest span for a metric does not have an end time, since it has not ended y
To get the spans on `my_tag_name` since 2025-05-14 10am, run:

```
for span in cvec.get_spans("mygroup/myedge/mode", start_at=datetime(2025, 5, 14, 10, 0, 0)):
for span in cvec.get_spans("mygroup/myedge/node", start_at=datetime(2025, 5, 14, 10, 0, 0)):
print("%s\t%s" % (span.value, span.raw_start_at))
```

Expand Down Expand Up @@ -115,13 +115,41 @@ Example output:
[46257 rows x 4 columns]
```

### Adding Metric Data

To add new metric data points, you create a list of `MetricDataPoint` objects and pass them to `add_metric_data`. Each `MetricDataPoint` should have a `name`, a `time`, and either a `value_double` (for numeric values) or a `value_string` (for string values).

```python
from datetime import datetime
from cvec.models import MetricDataPoint

# Assuming 'cvec' client is already initialized

# Create some data points
data_points = [
MetricDataPoint(
name="mygroup/myedge/compressor01/stage1/temp_out/c",
time=datetime(2025, 7, 29, 10, 0, 0),
value_double=25.5,
),
MetricDataPoint(
name="mygroup/myedge/compressor01/status",
time=datetime(2025, 7, 29, 10, 0, 5),
value_string="running",
),
]

# Add the data points to CVec
cvec.add_metric_data(data_points)
```

# CVec Class

The SDK provides an API client class named `CVec` with the following functions.

## `__init__(?host, ?tenant, ?api_key, ?default_start_at, ?default_end_at)`

Setup the SDK with the given host and API Key. The host and API key are loaded from environment variables CVEC_HOST, CVEC_TENANT, CVEC_API_KEY, if they are not given as arguments to the constructor. The `default_start_at` and `default_end_at` can provide a default query time interval for API methods.
Setup the SDK with the given host and API Key. The host and API key are loaded from environment variables CVEC_HOST, CVEC_API_KEY, if they are not given as arguments to the constructor. The `default_start_at` and `default_end_at` can provide a default query time interval for API methods.

## `get_spans(name, ?start_at, ?end_at, ?limit)`

Expand All @@ -143,6 +171,13 @@ If no relevant value changes are found, an empty list is returned.

Return all data-points within a given [`start_at`, `end_at`) interval, optionally selecting a given list of metric names. The return value is a Pandas DataFrame with four columns: name, time, value_double, value_string. One row is returned for each metric value transition.

## `add_metric_data(data_points, ?use_arrow)`

Add multiple metric data points to the database.

- `data_points`: A list of `MetricDataPoint` objects to add.
- `use_arrow`: An optional boolean. If `True`, data is sent to the server using the more efficient Apache Arrow format. This is recommended for large datasets. Defaults to `False`.

## `get_metrics(?start_at, ?end_at)`

Return a list of metrics that had at least one transition in the given [`start_at`, `end_at`) interval. All metrics are returned if no `start_at` and `end_at` are given.
6 changes: 1 addition & 5 deletions examples/add_metric_data_arrow_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,7 @@
def main() -> None:
cvec = CVec(
host=os.environ.get("CVEC_HOST", "https://your-subdomain.cvector.dev"),
email=os.environ.get("CVEC_EMAIL", "your-email@cvector.app"),
password=os.environ.get("CVEC_PASSWORD", "your-password"),
publishable_key=os.environ.get(
"CVEC_PUBLISHABLE_KEY", "your-cvec-publishable-key"
),
api_key=os.environ.get("CVEC_API_KEY", "your-api-key"),
)
test_metric_name = "python-sdk/test"
print("\nAdding new metric data using Arrow...")
Expand Down
6 changes: 1 addition & 5 deletions examples/add_metric_data_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,7 @@
def main() -> None:
cvec = CVec(
host=os.environ.get("CVEC_HOST", "https://your-subdomain.cvector.dev"),
email=os.environ.get("CVEC_EMAIL", "your-email@cvector.app"),
password=os.environ.get("CVEC_PASSWORD", "your-password"),
publishable_key=os.environ.get(
"CVEC_PUBLISHABLE_KEY", "your-cvec-publishable-key"
),
api_key=os.environ.get("CVEC_API_KEY", "your-api-key"),
)
test_metric_name = "python-sdk/test"
print("\nAdding new metric data...")
Expand Down
127 changes: 127 additions & 0 deletions examples/add_multiple_metrics_and_get_spans_for_them.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
import random
from cvec import CVec
from datetime import datetime, timedelta, timezone
import os
import io
import pyarrow.ipc as ipc # type: ignore[import-untyped]

from cvec.models.metric import MetricDataPoint


def main() -> None:
cvec = CVec(
host=os.environ.get("CVEC_HOST", "https://your-subdomain.cvector.dev"),
api_key=os.environ.get("CVEC_API_KEY", "your-api-key"),
)
test_metric_name = ""

# fetch & pick metrics
metrics = cvec.get_metrics(
start_at=datetime(2025, 7, 14, 10, 0, 0),
end_at=datetime(2025, 7, 14, 11, 0, 0),
)
print(f"Found {len(metrics)} metrics")
for metric in metrics:
print(f"- {metric.name} - {metric.id}")
if metrics:
found_metric_name = next((m.name for m in metrics if "Sensor_" in m.name))
assert found_metric_name, "No suitable metric found"
test_metric_name = found_metric_name
print(f"\nUsing metric: {test_metric_name}")

# Add metric non-Arrow data
random_number_nonarrow = random.randint(10000, 20000)
print(
f"\nAdding new metric data point with non-Arrow format for metric "
f"'{test_metric_name}' and values {random_number_nonarrow}..."
)
new_data = [
MetricDataPoint(
name=test_metric_name,
time=datetime.now(timezone.utc),
value_double=random_number_nonarrow,
value_string=None,
),
MetricDataPoint(
name=test_metric_name,
time=datetime.now(timezone.utc),
value_double=None,
value_string=str(random_number_nonarrow),
),
]
cvec.add_metric_data(new_data, use_arrow=False)
print("Non-Arrow Data added successfully")

# Add metric Arrow data

random_number_arrow = random.randint(10000, 20000)
print(
f"\nAdding new metric data point with Arrow format for metric "
f"'{test_metric_name}' and value {random_number_arrow}..."
)
new_data = [
MetricDataPoint(
name=test_metric_name,
time=datetime.now(timezone.utc),
value_double=random_number_arrow,
value_string=None,
),
]
cvec.add_metric_data(new_data, use_arrow=True)
print("Arrow Data added successfully")

# Fetch and print metric data - non-Arrow
data_points = cvec.get_metric_data(
start_at=datetime.now(timezone.utc) - timedelta(minutes=1),
end_at=datetime.now(timezone.utc),
names=[test_metric_name],
)
assert len(data_points) > 0, "No data points found for the metric"
assert any(dp.value_double == random_number_nonarrow for dp in data_points), (
"No data point found with the expected non-Arrow value"
)
assert any(dp.value_string == str(random_number_nonarrow) for dp in data_points), (
"No data point found with the expected non-Arrow string value"
)
assert any(dp.value_double == random_number_arrow for dp in data_points), (
"No data point found with the expected Arrow value"
)
print(f"\nFound {len(data_points)} data points for metric '{test_metric_name}'")
for point in data_points:
print(
f"- {point.name}: {point.value_double or point.value_string} at {point.time}"
)

# Fetch and print metric data - Arrow
arrow_data = cvec.get_metric_arrow(
start_at=datetime.now(timezone.utc) - timedelta(minutes=1),
end_at=datetime.now(timezone.utc),
names=[test_metric_name],
)
reader = ipc.open_file(io.BytesIO(arrow_data))
table = reader.read_all()
assert len(table) > 0, "No data found in Arrow format"
print(f"Arrow table shape: {len(table)} rows")
print("\nFirst few rows:")
for i in range(min(5, len(table))):
print(
f"- {table['name'][i].as_py()}: {table['value_double'][i].as_py() or table['value_string'][i].as_py()} at {table['time'][i].as_py()}"
)

# spans
spans = cvec.get_spans(
start_at=datetime.now(timezone.utc) - timedelta(minutes=1),
end_at=datetime.now(timezone.utc),
name=test_metric_name,
limit=5,
)
assert len(spans) > 0, "No spans found for the metric"
print(f"Found {len(spans)} spans")
for span in spans:
print(f"- Value: {span.value} from {span.raw_start_at} to {span.raw_end_at}")

print("\nAll operations completed successfully.")


if __name__ == "__main__":
main()
6 changes: 1 addition & 5 deletions examples/get_metric_arrow_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,7 @@
def main() -> None:
cvec = CVec(
host=os.environ.get("CVEC_HOST", "https://your-subdomain.cvector.dev"),
email=os.environ.get("CVEC_EMAIL", "your-email@cvector.app"),
password=os.environ.get("CVEC_PASSWORD", "your-password"),
publishable_key=os.environ.get(
"CVEC_PUBLISHABLE_KEY", "your-cvec-publishable-key"
),
api_key=os.environ.get("CVEC_API_KEY", "your-api-key"),
)
test_metric_name = "python-sdk/test"
print("\nGetting metric data as Arrow...")
Expand Down
6 changes: 1 addition & 5 deletions examples/get_metric_data_objects_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@
def main() -> None:
cvec = CVec(
host=os.environ.get("CVEC_HOST", "https://your-subdomain.cvector.dev"),
email=os.environ.get("CVEC_EMAIL", "your-email@cvector.app"),
password=os.environ.get("CVEC_PASSWORD", "your-password"),
publishable_key=os.environ.get(
"CVEC_PUBLISHABLE_KEY", "your-cvec-publishable-key"
),
api_key=os.environ.get("CVEC_API_KEY", "your-api-key"),
)
test_metric_name = "python-sdk/test"
print("\nGetting metric data as objects...")
Expand Down
6 changes: 1 addition & 5 deletions examples/get_metrics_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,7 @@ def main() -> None:
host=os.environ.get(
"CVEC_HOST", "https://your-subdomain.cvector.dev"
), # Replace with your API host
email=os.environ.get("CVEC_EMAIL", "your-email@cvector.app"),
password=os.environ.get("CVEC_PASSWORD", "your-password"),
publishable_key=os.environ.get(
"CVEC_PUBLISHABLE_KEY", "your-cvec-publishable-key"
),
api_key=os.environ.get("CVEC_API_KEY", "your-api-key"),
)
print("\nGetting available metrics...")
metrics = cvec.get_metrics()
Expand Down
6 changes: 1 addition & 5 deletions examples/get_spans_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@
def main() -> None:
cvec = CVec(
host=os.environ.get("CVEC_HOST", "https://your-subdomain.cvector.dev"),
email=os.environ.get("CVEC_EMAIL", "your-email@cvector.app"),
password=os.environ.get("CVEC_PASSWORD", "your-password"),
publishable_key=os.environ.get(
"CVEC_PUBLISHABLE_KEY", "your-cvec-publishable-key"
),
api_key=os.environ.get("CVEC_API_KEY", "your-api-key"),
)
metrics = cvec.get_metrics()
if metrics:
Expand Down
Loading