Log Analyzer

A utility for parsing, filtering, and analyzing log files efficiently.

Features

• Parse multiple log formats
• Filter logs by level, timestamp, or keyword
• Generate summary reports
• Export results to database

Requirements

• pyenv (or other python 3.12 installation)
• duckdb cli
• See requirements.txt for dependencies

Installation

Setup the dependencies

brew install pyenv
brew install duckdb

Setup the projects

git clone https://github.com/selfdanielj/log_analyzer.git
cd log_analyzer
pyenv exec python -m venv venv
python -m pip install -r requirements.txt

Usage

Run the process to download files and load to db

python analyze.py <proxy-prefix-name> <log-date-in-YYYY-MM-DD>

Start the duckdb-ui and query the data

duckdb -ui databases/prod_lbs_logs.duckdb