Paper to Obsidian

Turn dense research papers into beautiful, interconnected Obsidian notes locally on your Mac.

Paper to Obsidian is an AI-powered tool that transforms academic research papers into richly formatted Obsidian markdown notes. It uses a local Mixture-of-Experts pipeline to analyze every section of a paper through three distinct lenses, making complex research accessible at multiple levels of comprehension—all running 100% offline on Apple Silicon.

Features

• Multi-Perspective Analysis: Multi-Perspective Analysis: Instead of one generic summary, every section is analyzed by three fine-tuned expert adapters:

  • ELI5: Simple, analogy-based explanations for beginners
  • Intuitive: First-principles engineering explanations (Feynman-style)
  • Executive: Business/utility-focused summaries for quick understanding

• Vision AI Captioning: Uses Qwen2-VL to "see" charts, diagrams, and figures, embedding AI captions directly into your notes so the text model understands the visuals.

• Knowledge Graph Builder: Automatically extracts key concepts and wraps them in [[WikiLinks]], instantly connecting new papers to your existing Obsidian knowledge graph.

• Efficient Local Inference: Optimized sequential loading allows running 3+ LLMs on a standard 16GB MacBook Pro M2 without crashing.

Requirements

• Python 3.12+
• macOS with Apple Silicon (M1/M2/M3/M4)
• ~8GB RAM for model inference

Installation

# Clone the repository
git clone https://github.com/apatti/paper-to-obsidian.git
cd paper-to-obsidian

# Install dependencies using uv (recommended)
uv sync

# Or using pip
pip install -e .

Configuration

Create a .env file in the project root:

OBSIDIAN_VAULT_PATH=/path/to/your/obsidian/vault

Usage

python obsidian_paper.py /path/to/paper.pdf

This will:

1. Parse the PDF and extract all images
2. Caption figures using the vision model
3. Identify key concepts and technical terms
4. Generate three explanations per section (ELI5, Intuitive, Executive)
5. Output a formatted markdown file to your Obsidian vault

Output Example

Your Obsidian note will look like this:

# Paper Title

**Connected Concepts**: [[Transformer]] | [[Attention Mechanism]] | [[BERT]]

---

## Abstract

![Figure 1](assets/paper_name/image_0.png)
> [!INFO] AI Vision
> *A diagram showing the transformer architecture with encoder and decoder blocks...*

> [!TIP] ELI5 Analogy
> Imagine you're reading a book but you can look at any page instantly...

> [!EXAMPLE] Engineering Logic
> The key mechanism here is self-attention, which computes weighted sums...

> [!SUMMARY] Executive Verdict
> This approach reduces training time by 10x while maintaining accuracy...

Architecture

graph TD
    A[PDF Input] --> B[Layout Parsing\npymupdf4llm];
    B --> C[Vision Pass\nQwen2-VL-2B];
    C --> D{Matrix Analysis};
    D -->|Load| E[Executive Adapter];
    D -->|Load| F[Intuitive Adapter];
    D -->|Load| G[ELI5 Adapter];
    E & F & G --> H[Obsidian Vault];
    H --> I[Graph Connections];

Models Used

Component	Model	Function	Size
Base LLM	Llama 3.2 3B Instruct (4-bit)	The reasoning engine	~2GB
Vision	Qwen2-VL-2B Instruct (4-bit)	Image captioning	~1.5GB
Adapters	LoRA fine-tuned	The "Personas"	~8MB each

Fine-Tuning

See fineTune/README.md for instructions on creating custom adapters.

Project Structure

paper-to-obsidian/
|-- obsidian_paper.py       # Main entry point
|-- utils/
|   |-- vision.py           # Vision model captioning
|-- adapters/               # Fine-tuned LoRA adapters
|   |-- eli5_final/
|   |-- executive_final/
|   |-- intuitive_final/
|-- fineTune/               # Training pipeline
|-- .env                    # Configuration (create this)
|-- pyproject.toml          # Dependencies
|-- LICENSE                 # MIT License

Dependencies

Key dependencies include:

• mlx-lm / mlx-vlm: Apple Silicon ML inference
• pymupdf4llm: PDF to Markdown conversion
• python-dotenv: Environment configuration

See pyproject.toml for the complete list.

License

MIT License - see LICENSE for details.