From db85e4ef3b1b2668225e74d3d251c3c778b6434b Mon Sep 17 00:00:00 2001
From: newgnart <nvtrang91@gmail.com>
Date: Thu, 15 Jan 2026 23:40:48 -0800
Subject: [PATCH 1/2] Add testing, linting, and Kafka streaming documentation
 to CLAUDE.md

- Included sections for running tests and linting dbt models using SQLFluff.
- Documented Kafka streaming setup, including CLI entry points for initializing schemas, starting producers and consumers, and executing scripts directly.
- Enhanced overall project documentation for better clarity on testing and streaming processes.
---
 CLAUDE.md | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/CLAUDE.md b/CLAUDE.md
index 952c6ee..cc4e042 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -232,6 +232,54 @@ dbt docs generate
 dbt docs serve
 ```
 
+### Testing
+```bash
+# Run all tests
+uv run pytest
+
+# Run specific test file
+uv run pytest tests/test_connection.py
+
+# Run with verbose output
+uv run pytest -v
+```
+
+### Linting (dbt SQL)
+```bash
+# Lint dbt models with SQLFluff
+uv run sqlfluff lint dbt_project/models/
+
+# Fix auto-fixable issues
+uv run sqlfluff fix dbt_project/models/
+```
+
+### Kafka Streaming (Speed Layer)
+The project includes a Kafka-based streaming layer for near real-time event processing.
+
+**CLI Entry Points** (defined in pyproject.toml):
+```bash
+# Initialize Kafka schema/topics
+uv run kafka-init-schema
+
+# Start Kafka producer (streams events from HyperSync)
+uv run kafka-producer
+
+# Start Kafka consumer (writes to database)
+uv run kafka-consumer
+```
+
+**Direct Script Execution:**
+```bash
+# Producer - streams blockchain events to Kafka
+uv run python scripts/kafka/producer.py
+
+# Consumer - consumes from Kafka and loads to database
+uv run python scripts/kafka/consumer.py
+
+# Batch consumer (for Airflow integration)
+uv run python scripts/kafka/consume_batch.py
+```
+
 ## Environment Variables
 
 Required variables (see [.env.example](.env.example)):

From 4795d9715d209b6130d309c1c0eda8f187bec645 Mon Sep 17 00:00:00 2001
From: newgnart <nvtrang91@gmail.com>
Date: Fri, 16 Jan 2026 23:09:45 -0800
Subject: [PATCH 2/2] Remove unused graph visualization function from chat
 engine and update debug_conversion.sql comment for clarity

---
 chat_engine/chat_engine.py              | 11 -----------
 dbt_project/models/debug_conversion.sql |  2 +-
 2 files changed, 1 insertion(+), 12 deletions(-)

diff --git a/chat_engine/chat_engine.py b/chat_engine/chat_engine.py
index 0fff3c6..34e8d46 100644
--- a/chat_engine/chat_engine.py
+++ b/chat_engine/chat_engine.py
@@ -80,17 +80,6 @@ def chat_engine(query: str, enabled_agents: Optional[List[str]] = None):
     return graph.invoke(state)
 
 
-def visualize_graph():
-    """Display the graph structure (for Jupyter notebooks)."""
-    try:
-        from IPython.display import Image, display
-
-        graph = build_graph()
-        display(Image(graph.get_graph().draw_png()))
-    except ImportError:
-        print("IPython not available. Graph visualization skipped.")
-
-
 def main():
     """Main execution function with example queries."""
     # query1 = "What is openai's latest product?"
diff --git a/dbt_project/models/debug_conversion.sql b/dbt_project/models/debug_conversion.sql
index 915308e..9530b38 100644
--- a/dbt_project/models/debug_conversion.sql
+++ b/dbt_project/models/debug_conversion.sql
@@ -1,6 +1,6 @@
 {{ config(materialized='view') }}
 
--- Debug model to check hex conversion
+-- Debug model to check hex conversion for Transfer events
 -- Run: dbt run --select debug_conversion --target prod
 
 with sample_data as (