improve auto-generated skills for future reuse

[reneehuang1]

Currently, each session already auto-generates a skill. However, according to a use case I tried (getting a music list in Spotify), it failed often. I also find that the generated skills are probably not very useful for helping the AI prevent mistakes in the future.

I am not sure about the solution yet, but I want to note down the issue here.

Skill 2: "Get a music list in Spotify for music like Glass Animals"

  Session (session-20260222-170804) — 42 steps

  Model: gpt-5.3-codex

  This was a much more complex session with multiple obstacles:

  1. Steps 1-12: Attempt to install Spotify via Play Store — Opened Play Store, dismissed promos, searched "Spotify", found the
   app, but installation repeatedly failed with "Can't install Spotify" errors.
  2. Step 13: request_user_decision — Agent asked the user how to proceed. User chose "Use YouTube Music to build a similar
  list now".
  3. Steps 14-23: YouTube Music fallback — Dismissed the error, launched YouTube Music via explicit intent, bypassed premium
  trial popup and onboarding, searched "glass animals", and tapped "Mix" to open a Glass Animals artist mix.
  4. Steps 24-37: Second attempt to install Spotify — Despite the YouTube Music success, the agent circled back to try Spotify
  installation again via Play Store deep links. Hit the same "Can't install" error repeatedly.
  5. Step 35: Second request_user_decision — User provided custom input (29 chars, content not shown).
  6. Steps 37-41: Spotify Web fallback — Agent opened open.spotify.com/search/glass%20animals in Chrome, searched "glass
  animals radio", tapped the result.
  7. Step 42: finish — Successfully landed on the Glass Animals Radio playlist on Spotify Web, showing tracks like "The Other
  Side Of Paradise" and "Electric Love" with related artists (BORNS, alt-J, Cage The Elephant).

  Key takeaway: The agent encountered persistent Spotify install failures on the emulator, pivoted through YouTube Music, then
  ultimately succeeded via Spotify Web in Chrome — a 42-step session with multiple recovery strategies and two user decision
  points.

[tsubasakong]

/fix

[github-actions]

Summary
I dug into how auto skills are generated and surfaced. The current template doesn’t match the loader’s metadata rules, so summaries are low-signal, and each run creates a one-off skill file. That makes reuse unlikely and can even crowd out useful skills.

Probable Root Cause(s)

1. Auto-skill template puts - Generated: ... as the first non-heading line, which becomes the description in SkillLoader. This makes the prompt summary unhelpful and hurts skill selection. See src/skills/auto-artifact-builder.ts and src/skills/skill-loader.ts.
2. Auto-skill filename includes timestamp, so each run creates a unique id and never consolidates/updates. That prevents reuse and can flood the summary list (only first 20 skills are shown).
3. Auto-skill content is mostly raw action traces (tap, swipe, etc.) with no app preconditions, decision points, or fallbacks, which makes reuse poor for future sessions.

Likely Files/Modules

• src/skills/auto-artifact-builder.ts (template + file naming)
• src/skills/skill-loader.ts (metadata parsing rules)
• src/agent/agent-runtime.ts (when auto skills are created)
• Docs: frontend/tools/skills.md

Reproduction / Verification Steps

1. Run any successful task (e.g., the Spotify/Glass Animals example).
2. Inspect the generated auto skill in workspace/skills/auto/(timestamp)-(slug).md.
3. Observe that the first non-heading line is - Generated: ..., so the skill summary shows only the timestamp.
4. Run openpocket skills list or check the system prompt summary and notice that auto skills are low-signal and numerous.

Minimal Fix Options

1. Recommended: Change the auto-skill template to include a real one-line description immediately after the # heading, and make it the “why/how” of the task (not metadata). Example:
   Create a Glass Animals–like playlist; prefer Spotify app if installed, else Spotify Web.
   This directly fixes the prompt summary quality without changing loader behavior.
2. Make auto-skill file IDs stable (e.g., auto-(slug).md) and overwrite/update an existing file instead of creating a timestamped one every time. This improves reuse and avoids summary spam.
3. Add a quality gate before writing skills (e.g., only if ok and steps < N and no repeated install failures). This keeps only reusable flows.
4. Expand the template to include preconditions, decision points, and fallbacks derived from traces (e.g., “If Play Store install fails, open Spotify Web in Chrome”).

Recommended Approach
Start with option 1 + 2:

• Put a concise one-line description right after the title so it shows up in the skill summary.
• Use a stable filename to consolidate skills by task.
  Then optionally add option 4 (fallbacks/constraints) once the basic summary is high-signal.

Test Cases To Add/Update

• test/auto-artifact-builder.test.mjs: assert that the first non-heading line is a meaningful description (not metadata) and that SkillLoader picks it up.
• Add a test that auto skills with the same task overwrite/merge (if you adopt stable IDs).
• Add a test for gating logic if you add quality thresholds.

Risk / Rollback Notes

• Risk: Changing the template or IDs could alter which skills are surfaced in the system prompt and affect behavior. Keep changes minimal and revert by restoring the old template/filenames if needed.
• If you gate skill creation too aggressively, you may end up with no auto skills for some tasks.

Lightweight Checks Performed
Read src/skills/auto-artifact-builder.ts, src/skills/skill-loader.ts, and frontend/tools/skills.md to confirm how descriptions are parsed and how auto skills are written.

Questions

1. Should auto skills be overwritten per task (stable ID), or do you want a history of runs?
2. Do you want auto skills used by default, or should they be “opt-in” for prompt injection?
3. Any desired schema for the one-line description (e.g., “Goal + preferred path + fallback”)?

Generated by issue-fix for issue #32