Skip to content

Conversation

Copy link

Copilot AI commented Dec 12, 2025

Adds tooling to query Kubernetes pod and node inventory from Azure Log Analytics and analyze resource consumption by HPCC component.

Scripts Added

get_pod_node_inventory.py - Data collector

  • Queries KubePodInventory and KubeNodeInventory tables via Azure REST API with KQL
  • Filters by namespace and time range
  • Supports workspace ID or AKS cluster discovery
  • CSV output with metadata headers
  • RFC 1123 namespace validation with KQL injection protection

analyze_pod_node_inventory.py - Analyzer

  • Identifies HPCC components from pod naming conventions (dali, esp, thor-, roxie-, etc.)
  • Maps pods → nodes (computers) for each component
  • Calculates pod count, node count, duration, and pod-hours per component
  • CSV and text output formats with optional detailed breakdown

Usage

# Collect data
./get_pod_node_inventory.py \
  --workspace-id <id> -n hpcc \
  --start-time "2025-11-04 12:00" \
  --end-time "2025-11-04 18:00" > inventory.csv

# Analyze
./analyze_pod_node_inventory.py inventory.csv --format text --by-component

Output shows component resource consumption:

Component                    Pods    Nodes    Duration    Pod-Hours
dali                           1        1      6.00h        6.00h
thor-mycluster                 1        1      6.00h        6.00h
thor-mycluster-worker          8        8      6.00h       48.00h
roxie-cluster1                 4        4      6.00h       24.00h

Notes

Duration estimation assumes pods in snapshot ran for entire time window. For accurate pod lifecycle tracking, time-series data would be needed.

Follows established get/analyze pattern from get_file_access_logs.py and analyze_file_access.py.

Original prompt

Following the same get/analyze pattern and common parareters used in other azure/ scripts, createa 2 python scripts to: 1)

fetch pod information from KubePodInventory using az 'rest' and KQL
fetch node information from KubeNodeInventory using az rest and KQL
allow filtering by namespace and time range

An analysis script that takes the csv output(s) created by (1), and:
works out which pods belong to which HPCC components
cross-reference the pods -> computer(nodes)
workout how many pods/nodes each component consumed for how long


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 6 commits December 12, 2025 09:22
Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
…licate code

Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
Copilot AI changed the title [WIP] Add scripts to fetch and analyze Kubernetes pod and node data Add pod/node inventory scripts for Azure Log Analytics with HPCC component analysis Dec 12, 2025
Copilot AI requested a review from jakesmith December 12, 2025 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants