Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# GoReleaser configuration for ClippingKK CLI
version: 2

project_name: ck-cli

before:
hooks:
# Clean up any previous builds
- go mod tidy
# Run tests before building
- go test ./...

builds:
- main: ./cmd/ck-cli
binary: ck-cli
env:
- CGO_ENABLED=0
goos:
- linux
- windows
- darwin
goarch:
- amd64
- arm64
# Build flags
flags:
- -trimpath
ldflags:
- -s -w
- -X main.Version={{.Version}}
- -X main.Commit={{.ShortCommit}}
# Ignore specific combinations that don't make sense
ignore:
- goos: windows
goarch: arm64

archives:
- format: tar.gz
# Use zip for Windows
format_overrides:
- goos: windows
format: zip
name_template: >-
{{ .ProjectName }}_
{{- title .Os }}_
{{- if eq .Arch "amd64" }}x86_64
{{- else if eq .Arch "386" }}i386
{{- else }}{{ .Arch }}{{ end }}
{{- if .Arm }}v{{ .Arm }}{{ end }}
files:
- README.md
- LICENSE

checksum:
name_template: 'checksums.txt'

snapshot:
name_template: "{{ incpatch .Version }}-next"

changelog:
sort: asc
use: github
filters:
exclude:
- '^docs:'
- '^test:'
- '^ci:'
- '^chore:'
- Merge pull request
- Merge branch
groups:
- title: 'New Features'
regexp: '^.*?feat(\(.+\))??!?:.+$'
order: 0
- title: 'Bug Fixes'
regexp: '^.*?fix(\(.+\))??!?:.+$'
order: 1
- title: 'Performance Improvements'
regexp: '^.*?perf(\(.+\))??!?:.+$'
order: 2
- title: 'Refactors'
regexp: '^.*?refactor(\(.+\))??!?:.+$'
order: 3
- title: 'Others'
order: 999

release:
github:
owner: clippingkk
name: cli
draft: false
prerelease: auto
mode: replace
header: |
## ClippingKK CLI {{.Tag}}

Parse Amazon Kindle clippings and sync to ClippingKK service.

### Installation

Download the appropriate binary for your platform from the assets below.

### What's Changed
footer: |
**Full Changelog**: https://github.com/clippingkk/cli/compare/{{ .PreviousTag }}...{{ .Tag }}

184 changes: 137 additions & 47 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,81 +4,136 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## Project Overview

ClippingKK CLI (`ck-cli`) is a Rust-based Terminal User Interface tool that parses Amazon Kindle's "My Clippings.txt" file into structured JSON format. It supports synchronization with the ClippingKK web service for cloud storage of reading highlights.
ClippingKK CLI (`ck-cli`) is a Go-based command-line tool that parses Amazon Kindle's "My Clippings.txt" file into structured JSON format. It supports synchronization with the ClippingKK web service for cloud storage of reading highlights.

## Key Commands

### Building and Development
```bash
# Standard build
cargo build
make build
# or
go build -o ck-cli ./cmd/ck-cli

# Release build (optimized)
cargo build --release
# Install to GOPATH/bin
make install
# or
go install ./cmd/ck-cli

# Run tests
cargo test
make test
# or
go test ./...

# Run benchmarks
cargo bench
# Run tests with coverage
make test-coverage

# Format code
cargo fmt
make fmt
# or
go fmt ./...

# Lint code
cargo clippy --all-features
make lint
# or
golangci-lint run

# Run the CLI
cargo run -- [arguments]
./ck-cli [arguments]
# or
go run ./cmd/ck-cli [arguments]
```

### Testing
### Cross-Platform Building
```bash
# Run all tests with verbose output
cargo test --verbose --all-features --workspace
# Build for all platforms
make build-all

# Build for specific platforms
make build-linux
make build-windows
make build-macos

# Release with GoReleaser
make release-dry # dry run
make release # actual release
```

### Testing and Development
```bash
# Run all tests
go test ./...

# Run tests with verbose output
go test -v ./...

# Run specific test
cargo test test_name
go test -run TestName ./internal/parser

# Generate code coverage (requires cargo-tarpaulin)
cargo +nightly tarpaulin --verbose --all-features --workspace --timeout 120 --out Xml
# Run benchmarks
make bench
# or
go test -bench=. ./...

# Test with example data
make run-example
make test-parse-stdin
```

### Running the Application
```bash
# Parse a Kindle clippings file to JSON
cargo run -- parse -i "/path/to/My Clippings.txt" -o "/path/output.json"
ck-cli parse --input "/path/to/My Clippings.txt" --output "/path/output.json"

# Parse from stdin to stdout
cat "My Clippings.txt" | cargo run -- parse
cat "My Clippings.txt" | ck-cli parse

# Sync to ClippingKK web service (requires login first)
cargo run -- login --token "YOUR_TOKEN"
cargo run -- parse --input "/path/to/My Clippings.txt" --output http
ck-cli login --token "YOUR_TOKEN"
ck-cli parse --input "/path/to/My Clippings.txt" --output http
```

## Architecture and Code Structure

### Project Structure
```
cmd/ck-cli/ # Main CLI application entry point
internal/
├── commands/ # CLI command implementations (login, parse)
├── config/ # Configuration management (TOML files)
├── http/ # HTTP client and GraphQL integration
├── models/ # Data models (ClippingItem, etc.)
└── parser/ # Kindle clippings parser (core logic)
```

### Core Components

1. **Parser Module (`src/parser.rs`)**: The heart of the application
- Handles multi-language parsing (Chinese, English, Japanese)
- Uses regex patterns to extract clipping components
- Converts Kindle's format to `TClippingItem` structs
- Key struct: `TClippingItem` with fields: `title`, `content`, `pageAt`, `createdAt`
1. **Main CLI (`cmd/ck-cli/main.go`)**: Application entry point
- Uses `urfave/cli/v2` framework for command structure
- Handles graceful shutdown with context
- Version and build info injection

2. **Parser Module (`internal/parser/parser.go`)**: The heart of the application
- Handles multi-language parsing (Chinese, English)
- Uses regex patterns to extract clipping components
- Converts Kindle's format to `ClippingItem` structs
- Key struct: `ClippingItem` with fields: `Title`, `Content`, `PageAt`, `CreatedAt`

2. **HTTP/GraphQL Integration (`src/http.rs`, `src/graphql.rs`)**:
3. **HTTP/GraphQL Integration (`internal/http/client.go`)**:
- Syncs parsed clippings to ClippingKK web service
- Uses GraphQL mutations for data upload
- Handles authentication via Bearer tokens
- Handles chunked uploads with concurrency control
- Proper error handling and retry logic

3. **Configuration (`src/config.rs`)**:
4. **Configuration (`internal/config/config.go`)**:
- Manages `.ck-cli.toml` in user's home directory
- Stores HTTP endpoint and authentication headers
- TOML format with `pelletier/go-toml/v2`

4. **Authentication (`src/auth.rs`)**:
- Interactive login flow with QR code display
- Token management for API access
5. **Commands (`internal/commands/`)**:
- `login.go`: Authentication flow and token management
- `parse.go`: Main parsing and output logic
- Clean separation of CLI logic from business logic

### Data Flow
1. Input: Kindle's "My Clippings.txt" file (UTF-8 encoded)
Expand All @@ -87,31 +142,66 @@ cargo run -- parse --input "/path/to/My Clippings.txt" --output http

### Key Technical Details

- **Async Runtime**: Uses Tokio for async operations
- **Error Handling**: Returns `Result<T, Box<dyn Error>>` for main operations
- **Date Parsing**: Handles multiple date formats across languages using chrono
- **CLI Framework**: Uses `urfave/cli/v2` for robust command handling
- **HTTP Client**: Custom HTTP client with proper context handling
- **Concurrency**: Controlled concurrent uploads with semaphores
- **Error Handling**: Structured error handling with context
- **Date Parsing**: Handles multiple date formats across languages
- **Regex Patterns**: Language-specific patterns for parsing clipping headers
- **JSON Serialization**: Uses serde for type-safe JSON handling
- **JSON Serialization**: Standard library JSON with custom marshaling

### Testing Approach

- Unit tests in `tests/tests.rs` validate parsing against fixture files
- Fixtures cover edge cases: multiple languages, special characters, various formats
- Benchmarks in `benches/parse.rs` measure parsing performance using Criterion
- Unit tests in `*_test.go` files alongside source code
- Test fixtures cover edge cases: multiple languages, special characters, various formats
- Table-driven tests for comprehensive coverage
- Integration tests for command-line interface

### Build and Release

- **GoReleaser**: Multi-platform builds and releases
- **Docker**: Container builds for easy deployment
- **Package Managers**: Homebrew, APT, RPM, AUR support
- **CI/CD**: GitHub Actions for testing and releases

### CI/CD Workflows
### Dependencies

1. **Code Coverage**: Runs on master branch and PRs, reports to Codecov
2. **Benchmark Comparison**: Compares performance metrics on PRs
3. **Release Builds**: Automated multi-platform builds for releases
- `github.com/urfave/cli/v2`: CLI framework
- `github.com/pelletier/go-toml/v2`: TOML configuration
- Standard library for HTTP, JSON, regex, time handling

### Important Patterns

- The parser uses a state machine approach to handle multi-line clippings
- Language detection is based on regex pattern matching of clipping headers
- HTTP client reuses connections for batch uploads
- Configuration persists between sessions in TOML format
- Clean architecture with internal packages
- Interface-based design for testability
- Context-aware operations for cancellation
- Structured logging and error handling
- Configuration with sensible defaults

## Development Guidelines

### Code Style
- Follow standard Go conventions (`gofmt`, `go vet`)
- Use `golangci-lint` for comprehensive linting
- Write tests for all new functionality
- Document exported functions and types

### Testing
- Write table-driven tests where appropriate
- Include both positive and negative test cases
- Test error conditions and edge cases
- Use test fixtures for parser testing

### Performance
- Parser optimized for large clipping files
- Concurrent HTTP uploads for better sync performance
- Minimal memory allocation in hot paths

## Commit Guidelines

- The commit message must fit with Conventional Commits rules, and follow scopes (feat, fix, refactor, perf)
- The commit message must fit with Conventional Commits rules
- Use scopes: `feat`, `fix`, `refactor`, `perf`, `test`, `docs`, `build`
- Examples:
- `feat(parser): add support for Japanese clippings`
- `fix(http): handle network timeouts properly`
- `refactor(config): simplify TOML configuration`
35 changes: 35 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Build stage
FROM golang:1.21-alpine AS builder

# Install ca-certificates for SSL
RUN apk add --no-cache ca-certificates git

# Set working directory
WORKDIR /app

# Copy go mod files
COPY go.mod go.sum ./

# Download dependencies
RUN go mod download

# Copy source code
COPY . .

# Build the application
RUN CGO_ENABLED=0 GOOS=linux go build \
-ldflags="-s -w" \
-o ck-cli \
./cmd/ck-cli

# Final stage
FROM scratch

# Copy ca-certificates from builder
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Copy the binary
COPY --from=builder /app/ck-cli /ck-cli

# Set the entrypoint
ENTRYPOINT ["/ck-cli"]
Loading
Loading