Log Fetcher Tools

This repository contains Python scripts to fetch logs from Elasticsearch and Loki (Graphana Loki) in a convenient, interactive way.

Tools Included

1. es_log_fetcher.py: Interactively fetch logs from an Elasticsearch cluster. Supports searching by namespace, date range, and pod name keyword. It mimics the behavior of Loki's log retrieval but for ES indices.
2. loki_log_fetcher.py: Interactively fetch logs from a Loki instance using LogQL. Supports namespace selection, date ranges (local or UTC), app keyword filtering, and container selection.

Prerequisites

• Python 3.x
• requests library

Installation

1. Clone this repository:

   git clone https://github.com/cracya01/log-fetcher-tools.git
   cd log-fetcher-tools

2. Install dependencies:

   pip install requests

Configuration

Before running the scripts, you MUST update the configuration variables at the top of each script to match your environment.

es_log_fetcher.py

Open the file and edit the following lines:

# 1. Configuration - PLEASE UPDATE THESE
es_host = "https://your-es-host.com" # Replace with your ES URL
auth = ("your_username", "your_password") # Replace with your ES credentials

loki_log_fetcher.py

Open the file and edit the following lines:

# Configuration - PLEASE UPDATE THIS
url = "http://your-loki-host:3100" # Replace with your Loki URL

Usage

Using Elasticsearch Fetcher

Run the script:

python es_log_fetcher.py

Follow the interactive prompts:

1. Namespace: Enter the Kubernetes namespace (e.g., dev).
2. Date: Enter a single date (2025.12.01) or a range (2025.12.01-2025.12.03).
3. App Name Keyword: Enter a keyword to search for specific pods (e.g., api).
4. Log Count: Enter the number of logs to retrieve or all.
5. Output: Choose to print to console or save to a file.

Using Loki Fetcher

Run the script:

python loki_log_fetcher.py

Follow the interactive prompts:

1. Namespace: Enter the Kubernetes namespace.
2. Date: Enter date or range. You can also successfully choose between Local Time and UTC.
3. App Name Keyword: Filter pods by name.
4. Select Pods: Choose specific pods from the list.
5. Select Containers: Choose specific containers.
6. Log Count & Output: Similar to the ES fetcher.

License

MIT