lineate

Paste a link to an article, PDF, video, or thread and Lineate produces a private GitHub Gist that helps you prioritise what to read and internalise what you do read. It starts with takeaways, a devil’s-advocate critique, and a personalised “skip this” rationale, then includes the content or a semantically complete summary.

What it does

• You can paste a link (or any text containing links) and get back cleaned and converted URLs.
• Lineate can fetch content or transcripts, write them to a private GitHub Gist, and return the gist URL.
• Lineate can also rewrite URLs to more extraction-friendly forms.
• Processed URL(s) are opened in your browser by default.

Epistemic triage prefix

Every output starts with a short triage header that helps you prioritise what to read and engage more deeply with what you do read.

• The main takeaways capture the central claims and conclusions.
• A 60 word devil’s-advocate critique pressures the claims and assumptions.
• A 30 word “skip this” argument is written against your stated interests/persona. Persona configuration lives in PERSONA.txt.

Example output: https://gist.github.com/distbit0/4125df79e8a43b002a2d3a805b90e115

Supported inputs and adapters include: You can paste any text containing URLs, or a local file path for .pdf, .mp3, or .mp4.

• Some converters only run when you pass --force-convert-all or append ## to a specific URL.
• Converters that write gists include YouTube (/watch?, /live/), Twitter/X (/status/), Discord (discord.com), Telegram (t.me), PDFs (.pdf, arxiv.org/pdf/...), DocSend (docsend.com/view/...), GitBook (gitbook, docs.), Discourse (/t/...), Substack, Apple Podcasts, SoundCloud, Streameth, Rumble, and direct .mp3/.mp4.
• URL normalisers include Medium (medium.com to scribe.rip), Google Docs export, Wikipedia mobile, Reddit (old.reddit.com), and GreaterWrong (lesswrong.com).
• Warpcast (warpcast.com) is passed through unchanged.

Quick start

1. Install Python deps (managed via pyproject.toml):
   • uv sync
2. Ensure external tools are installed (see below).
3. Fill in .env with required keys.
4. Run:
   • uv run --env-file .env -m lineate "https://..."
   • or just run with no args to use the clipboard.

Usage

uv run --env-file .env -m lineate [text-or-url]

Options:

• --force-convert-all forces conversion for all URLs.
• --summarise replaces the extracted body with a semantically complete summary.
• --no-open prevents opening processed URLs in your browser.
• --force-refresh forces refresh for all converters (equivalent to appending ###).
• --force-no-convert skips conversion for all URLs.

Environment variables (.env)

Put these in this repo’s .env

Required for core functionality:

• gh_api_key – GitHub personal access token with Gist scope (used by writeGist).

Required for audio/video transcription and summarisation:

• OPENAI_API_KEY – used for Whisper transcription and GPT summarization.

Required for specific sources:

• Discord: DISCORD_AUTH_TOKEN
• Telegram: TELEGRAM_API_ID, TELEGRAM_API_HASH, TELEGRAM_SESSION_NAME
• Twitter/X: TWITTER_BEARER_TOKEN, TWITTER_CT0_TOKEN, TWITTER_COOKIE
  • Optional: TWITTER_USER_AGENT, TWITTER_XCLIENTTXID, TWITTER_XCLIENTUUID

Notes:

• Telethon will create a .session file named after TELEGRAM_SESSION_NAME in the repo.
• Twitter/X credentials need to be refreshed if the session expires.

How to get each key (links + minimal steps):

• gh_api_key: create a GitHub personal access token (fine-grained or classic). For fine-grained, grant User permissions → Gists (read/write). Docs: Creating a personal access token, Gists permission scope.
• OPENAI_API_KEY: create/view your key in the OpenAI API keys page. Doc: Where do I find my OpenAI API key?.
• Telegram (TELEGRAM_API_ID, TELEGRAM_API_HASH): create an app on Telegram’s developer portal and copy the values from API development tools. Doc: Creating your Telegram Application.
• Discord (DISCORD_AUTH_TOKEN) — unofficial: extract your user token from the browser’s developer tools (Network → request headers → Authorization). Doc: discordpy-self “Authenticating”.
• Twitter/X (TWITTER_BEARER_TOKEN, TWITTER_CT0_TOKEN, TWITTER_COOKIE) — unofficial: use browser devtools on x.com to copy cookies (auth_token, ct0) and the request Authorization: Bearer ... header. Docs: Export X cookies (auth_token, ct0), Extract Bearer token from request headers.

System dependencies

• uv – Python package manager used to sync/install deps.
• ffmpeg – required by pydub and yt-dlp for audio conversion.
• yt-dlp – required for some sources (e.g., Rumble).
• Clipboard helper for pyperclip:
  • Linux (X11): xclip or xsel (or wl-clipboard on Wayland)
  • macOS: pbcopy/pbpaste (built-in)

Install (CLI dependencies):

• macOS (Homebrew):
  • brew install ffmpeg yt-dlp uv
• Linux (Fedora):
  • sudo dnf install ffmpeg yt-dlp uv

Python dependencies used by lineate

Installed via uv sync from pyproject.toml. Key runtime packages:

• requests, loguru, python-dotenv, pyperclip
• openai (Whisper transcription)
• pydub (audio slicing)
• youtube-transcript-api, bs4
• telethon (Telegram)
• yt-dlp (Rumble)
• soundcloud-lib
• python-dateutil (Discord timestamps)