Pocket Botwin

Pocket Botwin is a chat-first Android operator wired into local tools, Jetsons, and external APIs. It’s the primary client for the Botwin.tokyo ecosystem: a local-first, privacy-focused personal AI stack that treats your phone as the front-end to your own infrastructure.

---

Overview

Pocket Botwin is an Android app built around a single idea: chat is the control panel. The app routes natural language into structured tool calls, talks to local and remote services, and returns summarized results back into the conversation.

Core Concepts

• Chat-first UI using Jetpack Compose
• Tool-tag system (for example <<DnsTool>>example.com<</DnsTool>>) for calling tools
• Integration with Jetson Orchestrator (health, DBs, jobs, etc.)
• Local-first design with minimal external dependencies and explicit, controlled use of cloud services
• Modular “rooms” (Chat, Calendar, Meeting Room, Excel, Notes, Todo, connector rooms, etc.) that each own a focused slice of functionality

Pocket Botwin is opinionated: it’s built to run against hardware and services you control (Jetsons, servers, local networks), with SaaS treated as an optional accelerator—not the center of gravity.

---

Architecture

At a high level, Pocket Botwin is split into four layers:

UI Layer

Jetpack Compose screens for chat and feature “rooms” (calendar, meeting room, excel-style editor, connector UIs, etc.).
Each room is typically its own feature-* module with its own navigation entry point.

Tool Layer

Tool parsing and dispatch (ToolCaller, ToolHandlers). This layer is responsible for:

• Parsing tool tags emitted by the LLM
• Mapping tags to strongly typed handlers
• Managing round-trip calls to APIs, Jetson services, and local capabilities
• Returning structured results to be summarized back into chat

Core Libraries

Shared core-* modules that handle cross-cutting concerns:

• Networking and HTTP clients
• Prompt storage and retrieval
• TTS and STT engines and queues
• Notifications and foreground services
• Secrets and configuration
• DB access
• Haptics and sound

Services

External systems Pocket Botwin talks to, including:

• Jetson Orchestrator APIs (system, health, DBs, jobs, etc.)
• OpenAI (LLM and tools)
• Other third-party APIs (Etherscan, price feeds, etc.)

---

Example Tool Call Flow

1. User message in chat
   “What’s the DNS info for botwin.tokyo?”

2. LLM emits a tool tag

   <<DnsTool>>botwin.tokyo<</DnsTool>>
   

3. Tool parsing and dispatch
   ToolCaller parses the tag and invokes the corresponding DnsToolHandler.

4. Handler execution
   The handler performs the network/system calls and returns structured results.

5. Summarization back to the user
   A follow-up LLM call summarizes the tool output into a user-facing reply, which is rendered in the chat UI (usually as both plain text and a structured card).

This pattern is shared across all tools (DNS, EthGas, EthTx, TradingView, Jetson health/DBs, flashcards, weather, etc.), so once you understand one tool’s lifecycle, you understand most of them.

---

Repo Structure

This section reflects the actual project layout from the tree you generated, minus Gradle caches and build outputs.

Root Files

• .gitignore
• build.gradle.kts
• gradle.properties
• gradlew
• gradlew.bat
• local.properties (machine-specific, not committed)
• project-structure.txt / project_structure.txt / project_tree.txt (diagnostic / structure dumps)
• settings.gradle.kts

---

Top-Level Modules

Each module uses standard Gradle/Android layout:

• src/main/java
• src/main/res
• src/androidTest/java
• src/test/java

---

app/

Android application module and shell: splash, top-level navigation, theming.

Key source roots:

• app/src/main/java/dev/sandbox/pocketbotwin_ui
  • shell – app shell / entry
  • splash – splash screen flows
  • theme – colors / typography / theming
  • ui and ui/theme – shared UI components and themes
• app/src/main/res
  • drawable
  • mipmap-anydpi-v26, mipmap-hdpi, mipmap-mdpi, mipmap-xhdpi, mipmap-xxhdpi, mipmap-xxxhdpi
  • values
  • xml
• app/src/androidTest/java/dev/sandbox/pocketbotwin_ui
• app/src/test/java/dev/sandbox/pocketbotwin_ui

---

main-room-chat/

Chat experience and “main room” where tool calls show up.

Source roots:

• main-room-chat/src/main/java/dev/sandbox/main/room/chat
  • assembler – wiring for messages + tool outputs
  • components – composables for bubbles, cards, tool output
  • contacts – contact-related UI hooks
  • dialogs – settings / automation / message-related dialogs
  • gallery – media / file attachment support
  • haptics – haptic feedback wiring
  • model – chat message, tool result, and state models
  • prefs – chat and room-level preferences
  • router – navigation / routing
  • secrets – secret-management hooks for chat
• Standard src/androidTest and src/test trees under dev/sandbox/main/room/chat.

---

feature-calendar-room/

Calendar UI, time-slot model, and alarm/reminder plumbing.

Source roots:

• feature-calendar-room/src/main/java/dev/sandbox/feature/calendar/room
  • Calendar screens, viewmodels, alarm scheduler, bootstrapper, etc.
• feature-calendar-room/src/androidTest/java/...
• feature-calendar-room/src/test/java/...

---

feature-meeting-room/

Meeting room: STT pipeline, transcription display, and “save to text” logic.

Source roots:

• feature-meeting-room/src/main/java/dev/sandbox/feature/meeting/room
  • UI, state, integration with STT, export to text file.
• Standard src/androidTest and src/test packages.

---

feature-excel-room/

Excel-like grid editor / “sheet” style room for structured data.

Source roots:

• feature-excel-room/src/main/java/dev/sandbox/feature/excel/room
  • Grid, cell state, editing, navigation, etc.
• Standard src/androidTest and src/test trees.

---

feature-takenote-room/

Note-taking room / text workspace.

Source roots:

• feature-takenote-room/src/main/java/dev/sandbox/feature/takenote/room
  • Note editor, list, state, etc.

---

feature-todo-room/

Todo / task room for simple task lists.

Source roots:

• feature-todo-room/src/main/java/dev/sandbox/feature/todo/room

---

feature-gdrive-room/

Google Drive integration UI.

Source roots:

• feature-gdrive-room/src/main/java/dev/sandbox/feature/gdrive/room

---

feature-gmail-room/

Gmail integration UI.

Source roots:

• feature-gmail-room/src/main/java/dev/sandbox/feature/gmail/room

---

feature-onedrive-room/

OneDrive integration room.

Source roots:

• feature-onedrive-room/src/main/java/dev/sandbox/feature/onedrive/room

---

feature-outlook-room/

Outlook integration room.

Source roots:

• feature-outlook-room/src/main/java/dev/sandbox/feature/outlook/room

---

feature-tool-caller/

Central ToolCaller + ToolHandler implementations.

Main source root:

• feature-tool-caller/src/main/java/dev/personalagent/feature/tool/caller
  • core – shared tooling for tool parsing, execution, result wrapping
  • tools – one subpackage per tool domain:
    • btc
    • calendar
    • coinprice
    • contacts
    • dns
    • eth
    • ethgas
    • ethtx
    • flashcard
    • gps
    • networktools
    • ocr
    • solana
    • takenote
    • todo
    • tradingview
    • weather
    • wikipedia

This is where the tag-to-handler mapping lives and where new tools are added.

---

room-shell/

Shared “room shell” infrastructure: navigation, windowing, and room management.

Main source root:

• room-shell/src/main/java/dev/sandbox/room/shell
  • grid
  • nav
  • registry
  • router
  • state
  • theme
  • window

---

core-contacts-secure/

Secure contact storage and models.

Main source root:

• core-contacts-secure/src/main/java/dev/personalagent/core/contacts/secure

---

core-db/

Shared DB layer.

Main source root:

• core-db/src/main/java/dev/personalagent/core/db

---

core-haptic-sounds/

Shared haptic + sound feedback utilities.

Main source root:

• core-haptic-sounds/src/main/java/dev/sandbox/core/haptic/sounds

---

core-llm/

Shared LLM integration layer.

Main source root:

• core-llm/src/main/java/dev/personalagent/core/llm

---

core-notifications/

Notification helper layer for foreground/background behavior.

Main source root:

• core-notifications/src/main/java/dev/sandbox/core/notifications

---

core-secrets/

Secrets handling, API tokens, and secure configuration access.

Main source root:

• core-secrets/src/main/java/dev/sandbox/core/secrets

---

core-stt/

Speech-to-text (STT) integration, including native pieces for whisper.

Main source root:

• core-stt/src/main/java/dev/personalagent/core/stt

---

core-tts/

Shared text-to-speech layer and TTS queueing.

Main source root:

• core-tts/src/main/java/dev/personalagent/core/tts

---

gradle/

Gradle wrapper support files:

• gradle/wrapper/gradle-wrapper.properties
• Wrapper JAR and related config.

---

That’s the actual module layout. All of the above have the expected src/androidTest/... and src/test/... package structures mirroring their src/main/... packages.

Project Details

Tooling & Versions

• Android Gradle Plugin (AGP): 8.12.3
• Kotlin: 2.0.0
• Kotlin Compose plugin: 2.0.0
• KSP: 2.0.0-1.0.24
• Gradle Java toolchain: JDK 17 (via Foojay resolver convention)
• JVM target: 17 for all Android and JVM modules

Android Configuration

• Namespace: dev.sandbox.app
• Application ID: dev.sandbox.app
• compileSdk: 34
• targetSdk: 34
• minSdk: 26
• Version name: 0.1.0
• Version code: 1

Global Gradle Settings

From gradle.properties:

• org.gradle.jvmargs = -Xmx2048m -Dfile.encoding=UTF-8
• android.useAndroidX = true
• kotlin.code.style = official
• android.nonTransitiveRClass = true

Toolchain and compiler settings:

• Kotlin Android + JVM modules use:
  • jvmTarget = 17
  • jvmToolchain(17) via subprojects { ... } in the root build file

Runtime Requirements

To build and run the project locally:

• Android Studio: Hedgehog or newer
• JDK: 17 (automatically provisioned via Gradle toolchains in most setups)
• Android SDK: 34 (with matching build tools installed)
• Device: Android 8.0 (API 26) or higher
  • Emulators are fine for basic flows; a physical device is recommended for full tool usage.

Key Libraries (App Module)

Core libraries used by :app:

• Jetpack Compose (BOM: androidx.compose:compose-bom:2024.10.01)
  • androidx.activity:activity-compose:1.9.2
  • androidx.compose.animation:animation
  • androidx.compose.ui:ui
  • androidx.compose.material3:material3
  • androidx.compose.foundation:foundation
  • androidx.compose.ui:ui-tooling-preview
  • androidx.compose.ui:ui-tooling (debug only)
• HTTP client: com.squareup.okhttp3:okhttp:4.12.0
• Security / crypto: androidx.security:security-crypto:1.1.0-alpha06
• Persistence / settings: androidx.datastore:datastore-preferences:1.1.1
• Lifecycle / ViewModel: androidx.lifecycle:lifecycle-viewmodel-compose:2.8.4

Internal Modules Wired into :app

The application module depends on the following internal modules:

• Shell / navigation:
  • :room-shell
  • :main-room-chat
• Feature rooms:
  • :feature-todo-room
  • :feature-takenote-room
  • :feature-calendar-room
  • :feature-meeting-room
  • :feature-gmail-room
  • :feature-outlook-room
  • :feature-gdrive-room
  • :feature-onedrive-room
  • :feature-excel-room
• Core infrastructure / tools:
  • :feature-tool-caller
  • :core-stt
  • :core-haptic-sounds
  • :core-secrets
  • :core-db
  • :core-llm
  • :core-notifications

Getting Started / Setup

Follow these steps to get Pocket Botwin running locally:

1. Clone the repository

   git clone git@github.com:botwin-tokyo/pocket-botwin-v1.git
   cd pocket-botwin-v1

2. Open in Android Studio

   • Launch Android Studio.
   • Choose File → Open… and select the project root (pocket-botwin-v1).
   • Let Android Studio import it as a Gradle project.

3. Let Gradle sync

   • Wait for the initial Gradle sync to complete.
   • Fix any missing SDK or build tools prompts (install SDK 34 if requested).

4. Create local.properties (if needed)

   If Android Studio didn’t generate it, create a local.properties file in the project root with your Android SDK path:

   sdk.dir=/path/to/Android/Sdk

   On Windows this will typically be something like:

    sdk.dir=C:\Users\<YourUser>\AppData\Local\Android\Sdk

5. Run on device or emulator

   • Plug in a physical Android device (API 26+), or start an emulator.
   • In Android Studio, select the app configuration.
   • Click Run ▶️ to install and launch Pocket Botwin.

---

Configuration & Secrets

In development, you have two main options:

1. Local Gradle/IDE configuration (build-time)
2. On-device secret storage (runtime) via core-secrets

A common pattern is:

• Add values to local.properties (never committed) and/or your IDE run configuration.
• Have core-secrets read those values and expose them to the rest of the app in a strongly-typed way.

Example local.properties entries (names are illustrative, not enforced):

OPENAI_API_KEY=sk-...
ETHERSCAN_API_KEY=...
JETSON_BASE_URL=https://api.botwin.tokyo
JETSON_BEARER_TOKEN=...

Important: Do not commit local.properties or any file containing secrets.

OpenAI, Etherscan, and Other APIs

The following keys are typically required for full functionality:

• OpenAI
  • Used by the LLM layer (core-llm) and any tools that rely on model calls.
• Etherscan
  • Used by ETH-related tools in feature-tool-caller (e.g., ethgas, ethtx, wallet/tx lookups).
• Other third-party APIs (price feeds, weather, etc.)
  • Each tool’s *ToolKeys.kt file documents which endpoints and keys it expects.

If a key is missing:

• The related tool handler should either no-op or return a “not configured” result.
• The LLM layer can then surface a friendly message instead of crashing the app.

Jetson Orchestrator Configuration

Pocket Botwin treats Jetson Orchestrator as a first-class backend for local/near-local operations.

Two primary values are required:

• Base URL – for example: https://api.botwin.tokyo
• Bearer token – a secret token used to authenticate to Jetson Orchestrator APIs.

In development, configure these via local.properties (or equivalent) and have core-secrets expose them to:

• feature-tool-caller tools that hit /v1/health, /v1/dbs, /v1/jobs, etc.
• Any future system/agent tools that rely on Jetson endpoints.

If the Jetson config is missing, the app should:

• Skip Jetson-specific tools, or
• Return a “Jetson not configured” status to the LLM for summarization.

Degradation Behavior (When Secrets Are Missing)

Tools generally fall into three buckets:

1. Local-only tools (no external key needed)

   • Example: DNS tools, basic network tools, some text-only utilities.
   • These continue to function without any cloud or Jetson config.

2. Cloud-key tools (OpenAI, Etherscan, etc.)

   • Example: ETH gas and transaction lookups, LLM-driven utilities.
   • If keys are missing:
     • Handler should return a structured “missing configuration” error.
     • The chat layer can display a clear message like “EthGas tool is not configured on this device.”

3. Jetson-backed tools

   • Example: Jetson health checks, DB/job tools, future agent tools.
   • If Jetson URL or token is missing:
     • The handler should treat Jetson as unavailable and avoid hitting the network.
     • The LLM can explain that Jetson orchestration is disabled or not configured.

The goal is that Pocket Botwin still boots and basic flows work even when only a subset of secrets are configured.

---

Build & Run Commands

If you prefer CLI or are wiring this into CI, you can build and test Pocket Botwin without opening Android Studio.

Build Debug APK

From the project root:

./gradlew assembleDebug

On Windows:

gradlew.bat assembleDebug

This produces a debug APK for the app module under:

• app/build/outputs/apk/debug/

Run Unit Tests

./gradlew test

This runs JVM unit tests across modules.

Run Instrumented Tests (Optional)

If you have device/emulator instrumented tests configured:

./gradlew connectedAndroidTest

Make sure an emulator or physical device is connected before running this.

---

Development Workflow / Branching

Pocket Botwin uses a simple, explicit branching model designed around feature branches and a stable main.

Branching Model

• main

  • Stable branch.
  • Should always build; used as the base for new features.

• feat/*

  • One branch per feature or tool.
  • Examples:
    • feat/dns-tool
    • feat/eth-gas-tool
    • feat/network-scan-tool
    • feat/ocr-attachments
    • feat/meeting-room-stt

Typical Flow

1. Sync main

   git checkout main
   git pull origin main

2. Create a feature branch

   git checkout -b feat/my-new-feature

3. Build and test locally

   ./gradlew assembleDebug
   ./gradlew test
   # and/or ./gradlew connectedAndroidTest

4. Merge main into your feature branch regularly

   git checkout feat/my-new-feature
   git merge main

5. Merge feature back into main when ready

   git checkout main
   git merge feat/my-new-feature
   git push origin main

You can do this via PRs or local merges, depending on how you’re managing the repo. The important part is: main stays clean and buildable, feature work lives on feat/* until it’s ready.

---

Tool & Tag System (High-Level)

Pocket Botwin’s tool system is built around simple, XML-like tags that the LLM emits and the app parses. This is the contract between the model and feature-tool-caller.

Generic Syntax

<<ToolName>>BODY<</ToolName>>

• ToolName matches the tag expected by a specific tool handler.
• BODY is a plain-text payload the handler knows how to interpret.

Examples

• DNS

  <<DnsTool>>botwin.tokyo<</DnsTool>>
  

• TradingView

  <<TradingView>>BTCUSD 1H<</TradingView>>
  

• EthGas

  <<ethgas>><</ethgas>>
  

Each of these corresponds to a handler under:

• feature-tool-caller/src/main/java/dev/personalagent/feature/tool/caller/tools/...

How It Works Internally

1. The LLM responds to a user message with a mix of normal text and one or more tool tags.
2. feature-tool-caller parses the response:
   • Extracts all <<...>>...<</...>> blocks.
   • Maps each tag to a specific handler using *ToolKeys.kt definitions.
3. The handler executes:
   • Reads the tag body.
   • Calls local or remote APIs (e.g., Jetson, Etherscan, DNS, etc.).
   • Returns a structured result object.
4. A “round 2” LLM call (using a dedicated summarization prompt) turns the raw result into a user-facing message and/or card in the chat UI.

Prompts and conventions for each tool live alongside its handler and *ToolKeys.kt, so adding or adjusting tools stays localized.

---

Ecosystem Context (Botwin.tokyo / Jetsons)

Pocket Botwin is not a standalone toy app; it’s the primary client for the Botwin.tokyo ecosystem and your Jetson-backed personal infrastructure.

Relationship to Botwin.tokyo

• Pocket Botwin – chat-first Android client, the “operator console” in your pocket.
• Botwin.tokyo – umbrella for documentation, research, and backend services.
• Jetson Orchestrator – FastAPI-style backend running on Jetsons or other local hardware, reachable at something like https://api.botwin.tokyo.

Pocket Botwin treats cloud services as optional accelerators. The default assumption is:

If it can run local, it should. If it can run on your hardware, even better.

Jetson Integration

Pocket Botwin talks to Jetson Orchestrator for:

• System health checks (/v1/health)
• Database introspection and queries (/v1/dbs/...)
• Job/task orchestration (/v1/jobs/...)
• Future agent endpoints for more autonomous flows

These are wired through tools in feature-tool-caller, using the Jetson base URL and bearer token provided by core-secrets.

If Jetson is offline or not configured, Pocket Botwin remains usable, but those specific tools will either be hidden or return a “not configured” summary.

Ryoko and the Agent Layer

Ryoko is the persona sitting on top of this stack: a local-first AI operator that:

• Reads and writes through Pocket Botwin.
• Calls tools via the tag system.
• Delegates heavier or scheduled work to Jetson Orchestrator.

You can think of Pocket Botwin as the UI and wiring, Ryoko as the personality and behavior, and the Jetsons as the muscle behind the scenes.

---

Support / Donations

Pocket Botwin is a passion project and all infrastructure, domain, and hardware costs are covered out of pocket by Botwin.tokyo. If you’re finding this useful and want to help keep the lights on (and the Jetsons humming), you can support the project here:

Crypto

• BTC: bc1ptljx4z5y8k7n3qp25jw06ythkf9uhc3elx9uwdg4ha82ck2ge0dsxchrl2
• ETH: 0xE66042e5C4A6ed912fDe025627120a849d8D6259
• Solana: 3pD5poCQf9zwnGBshC4kkyqU2JqUC7pzSjpPfXT9tmG3

Buy Me a Coffee

If you’d rather just toss a coffee instead of typing wallet addresses:

• Buy Me a Coffee(https://buymeacoffee.com/botwin)

License

Pocket Botwin is distributed under the Pocket Botwin Non-Commercial License v1.5.

• The full license text is available in the LICENSE file at the root of this repository.
• You may use, modify, and redistribute the software for non-commercial purposes only, subject to the terms of that license.
• Any Commercial Use (including SaaS/hosted offerings, resale, paid integrations, or using Pocket Botwin as part of a revenue-generating product or service) requires a separate written agreement with Botwin.tokyo.
• Use of the “Botwin”, “Pocket Botwin”, and “Botwin.tokyo” names or related marks is governed by the trademark rules in the LICENSE file and may not be used for commercial branding without prior written consent.