Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified .coverage
Binary file not shown.
22 changes: 12 additions & 10 deletions .env.solo.example
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
# Slopometry Solo-Leveler Configuration
# For basic session tracking and analysis
# Slopometry Solo Configuration
# For basic session tracking and code quality feedback

# Database Configuration
# Where to store session data (defaults to platform-specific location)
# SLOPOMETRY_DATABASE_PATH=/path/to/custom/slopometry.db

# Hook Configuration
# Command used by Claude Code hooks (auto-detected by default)
# SLOPOMETRY_HOOK_COMMAND=slopometry hook-handler

# Backup existing Claude settings before installing hooks
SLOPOMETRY_BACKUP_EXISTING_SETTINGS=true

# Stop Event Feedback
# Enable complexity analysis feedback when Claude Code sessions end
# This includes code health metrics vs previous commits for claude to review
SLOPOMETRY_ENABLE_STOP_FEEDBACK=True
# Complexity Analysis
# Enable code complexity analysis on session events
SLOPOMETRY_ENABLE_COMPLEXITY_ANALYSIS=true

# Enable stop hook feedback with code health metrics vs previous commits
# This gives Claude visibility into quality changes during the session
SLOPOMETRY_ENABLE_COMPLEXITY_FEEDBACK=true

# Include development guidelines from CLAUDE.md in feedback
SLOPOMETRY_FEEDBACK_DEV_GUIDELINES=false
48 changes: 24 additions & 24 deletions .env.summoner.example
Original file line number Diff line number Diff line change
@@ -1,37 +1,37 @@
# Slopometry Summoner Configuration
# For advanced experimentation with NFP Objectives and CLI
# Slopometry Summoner Configuration
# For advanced experimentation with LLM-based features
# Requires external LLM access - see docs/summoner.md for details

# Basic Configuration (inherited from solo)
SLOPOMETRY_DATABASE_PATH=/path/to/custom/slopometry.db
# SLOPOMETRY_DATABASE_PATH=/path/to/custom/slopometry.db
SLOPOMETRY_BACKUP_EXISTING_SETTINGS=true

# Depending on your experiment, you may not want to give hints to agents
# And set this False
SLOPOMETRY_ENABLE_STOP_FEEDBACK=True
# Complexity feedback (enabled by default)
SLOPOMETRY_ENABLE_COMPLEXITY_FEEDBACK=true
SLOPOMETRY_FEEDBACK_DEV_GUIDELINES=false

# LLM Integration for User Story Generation
# Available agents: claude-opus-4, gemini-2.5-pro, o3
SLOPOMETRY_USER_STORY_AGENTS=["claude-opus-4", "gemini-2.5-pro"]
# LLM Integration (required for userstorify and AI features)
# Set offline_mode=false to enable external LLM requests
SLOPOMETRY_OFFLINE_MODE=false
SLOPOMETRY_LLM_PROXY_URL=https://your-proxy.example.com
SLOPOMETRY_LLM_PROXY_API_KEY=your-api-key
SLOPOMETRY_LLM_RESPONSES_URL=https://your-proxy.example.com/responses

# User Story Generation
# Available agents: gpt_oss_120b, gemini
SLOPOMETRY_USER_STORY_AGENT=gpt_oss_120b

# Interactive Rating for Dataset Quality Control
# This will ask you to rate the user-stories generated by agents above
SLOPOMETRY_INTERACTIVE_RATING_ENABLED=true
# Prompts you to rate generated user stories (1-5)
SLOPOMETRY_INTERACTIVE_RATING_ENABLED=false

# Hugging Face Integration for Dataset Export
# Get your token from: https://huggingface.co/settings/tokens
SLOPOMETRY_HF_TOKEN=hf_your_token_here

# Default repository for dataset uploads (optional)
# Format: username/dataset-name
SLOPOMETRY_HF_DEFAULT_REPO=username/slopometry-dataset

# Experiment Configuration
# Maximum parallel workers for commit analysis experiments
SLOPOMETRY_MAX_EXPERIMENT_WORKERS=8

# Radon complexity analysis timeout (seconds)
SLOPOMETRY_RADON_TIMEOUT=30

# Git worktree cleanup (remove temporary directories after experiments)
# set this to false, if you need the code and not the benchmarks/datasets btw
SLOPOMETRY_CLEANUP_WORKTREES=true
# Performance Tuning
# Maximum parallel workers for file analysis
SLOPOMETRY_MAX_PARALLEL_WORKERS=6
# Maximum commits to analyze for baseline computation
SLOPOMETRY_BASELINE_MAX_COMMITS=100
67 changes: 32 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ A tool that lurks in the shadows, tracks and analyzes Claude Code sessions provi
*"Let's slop up all the things."*
— sherbie, opinionated SDET

# Features
# Features / FAQ

**NEWS:**

Expand All @@ -41,38 +41,59 @@ Please stop contacting us with your cries for mercy - this is between you and yo
![galen details](assets/galen_details.png)


### Eyeball progress based on overall session-vibes
### Q: How do i know if claude is lazy today?

A: Eyeball progress based on overall session-vibes

```bash
slopometry latest
```
<details>

Will show some metrics since the session start of the newest `claude code` session

![session statistics](assets/session-stat.png)

![complexity metrics (CC)](assets/cc.png)

### Q: I don't need to verify when my tests are passing, right?

A: lmao

Agents love to reward hack (I blame SWE-Bench, btw). Naive "unit-test passed" rewards teach the model to cheat by skipping them in clever ways.
What clevery ways you ask? Silent exception swallowing upstream ofc!

Slopometry forces agents to state the purpose of swallowed exceptions and skipped tests, this is a simple LLM-as-judge call for your RL pipeline (you're welcome)

### Dumb practices are now explicit and quantifiable!
Here is Opus 4.5, which is writing 90% of your production code by 2026:
![silent-errors](assets/force-review-silent-errors.png)
![silent-errors2](assets/force-review-silent-errors-2.png)

"> But tensor, i don't use slopometry and already committed to production!?"
Don't worry, your customers probably don't read their code either, and their agents will just run python -c "<1600 LOC adhoc fix>" as a workaround for each api call.

### Q: I am a junior and all my colleagues were replaced with AI before I learned good code taste, is this fine?

A: Here are some dumb practices agents love to add, that you should never show to anyone who cares about readable and predictable code:

![code_smells1](assets/code_smells1.png)

![code_smells2](assets/code_smells2.png)

### Measure your negative improvement since session start*!
### Q: I have been vibe-coding this codebase for a while now and learned prooompt engineering. Clearly the code is better now?

A: You're absolutely right (But we verify via code trends for the last ~100 commits anyway):

![session_delta](assets/session_delta.png)

*just got lucky here, plz ignore
### Q: I use Cursor/BillionDollarVSCodeForkFlavourOfTheWeek, which uses embeddings and RAG on my code in the cloud, so my agent always knows which file are related to the current task, right?

### Measure agent blind spots when vibe editing files before reading!
A: Haha, sure, maybe try a trillion dollar vscode fork, or a simple AST parser that checks imports for edited files and tests instead. Spend the quadrillions saved on funding researchers who read more than 0 SWE books during their careers next time.

![blind_spots](assets/blind_spots.png)


### Preserve incriminating evidence!
### Q: My boss is llm-pilled and asks me to report my progress every 5 minutes, but human rights forbid keylogging in my country, what do I do?
A: export your claude code transcripts and commit them into the codebase!

![evidence](assets/evidence.png)
**legal disclaimer**: transcripts are totally not for any kind of distillation, but merely for personal entertainment purposes
Expand Down Expand Up @@ -140,9 +161,6 @@ slopometry solo show <session_id>
# Alias for latest session, same as solo show <session_id>
slopometry latest

# Analyze the last 100 commits for trend analysis caching vs. current changes (can take a while)
slopometry summoner current-impact

# Save session artifacts (transcript, plans, todos) to .slopometry/<session_id>/
slopometry solo save-transcript # latest
slopometry solo save-transcript <session_id>
Expand Down Expand Up @@ -194,15 +212,11 @@ Slopometry can be configured using environment variables or a `.env` file:
# Create config directory and copy example config
mkdir -p ~/.config/slopometry

# For solo-leveler users (basic session tracking):
# Copy example config
curl -o ~/.config/slopometry/.env https://raw.githubusercontent.com/TensorTemplar/slopometry/main/.env.solo.example

# For summoner users (advanced experimentation):
curl -o ~/.config/slopometry/.env https://raw.githubusercontent.com/TensorTemplar/slopometry/main/.env.summoner.example

# Or if you have the repo cloned:
# cp .env.solo.example ~/.config/slopometry/.env
# cp .env.summoner.example ~/.config/slopometry/.env

# Edit ~/.config/slopometry/.env with your preferences
```
Expand All @@ -217,24 +231,6 @@ uv sync --extra dev
uv run pytest
```

### Running Tests with LLM Integration

By default, LLM integration tests are skipped because `offline_mode` is enabled. To run the full test suite including LLM tests:

```bash
# Set up credentials in .env (copy from example)
cp .env.summoner.example .env
# Edit .env with your LLM proxy credentials:
# - SLOPOMETRY_LLM_PROXY_URL
# - SLOPOMETRY_LLM_PROXY_API_KEY
# - SLOPOMETRY_LLM_RESPONSES_URL

# Run tests with offline mode disabled
SLOPOMETRY_OFFLINE_MODE=false uv run pytest tests/test_llm_integration.py -v
```

The integration tests make real API calls to configured LLM providers and verify that agents return valid responses.

Customize via `.env` file or environment variables:

- `SLOPOMETRY_DATABASE_PATH`: Custom database location (optional)
Expand Down Expand Up @@ -264,6 +260,7 @@ Customize via `.env` file or environment variables:
[x] - Actually make a package so people can install this
[ ] - Add hindsight-justified user stories with acceptance criteria based off of future commits
[x] - Add plan evolution log based on claude's todo shenanigans
[ ] - Rename the readme.md to wontreadme.md because have it takes more than 15 seconds or whatever the attention span is nowadays to read it all. Maybe make it all one giant picture? Anyway, stop talking to yourself in the roadmap.
[ ] - Finish git worktree-based [NFP-CLI](https://tensortemplar.substack.com/p/humans-are-no-longer-embodied-amortization) (TM) training objective implementation so complexity metrics can be used as additional process reward for training code agents
[ ] - Extend stop hook feedback with LLM-as-Judge to support guiding agents based on smells and style guide
[ ] - Not go bankrupt from having to maintain open source in my free time, no wait...
Binary file added assets/force-review-silent-errors-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/force-review-silent-errors.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading