Skill Usage Rates — Full-History Session Audit

alignment_status: review

Full-history read of Claude Code and Codex prompt history plus rich assistant transcripts (20,971 user prompts, Dec 10 2025 - Jul 4 2026) answering: with 184 active skills in this repo, at what rates do users invoke skills and how often do agents invoke or recommend them? Headline: direct user usage is still head-heavy (/ship alone is 13.6% of every Claude prompt; the top 3 user commands are 66.5% of all direct user skill invocations), while agents touch a much wider surface: 3,215 user-authored repo-skill invocations versus 9,125 assistant-side skill contract-load events (3,747 distinct session-skill pairs) and 1,267 assistant recommendation/use-reference events. The largest adoption gap remains habit, not coverage: ~550 hand-typed "why is..." prompts shadow /investigate, which was directly invoked 166 times.

1. Overview Stats

Method: a user skill invocation is counted as a leading /command (Claude history.jsonl display field) or leading //$command (Codex history.jsonl text field), matched against the 184 active skill directories under packs/*/{claude,codex}/*/SKILL.md. Built-in harness commands (/clear, /usage, /resume, /model, /pack...) are separated out and not counted as repo-skill usage. Agent-side counts come from assistant records in ~/.claude/projects/**/*.jsonl and ~/.codex/sessions/**/*.jsonl: exact skill contract loads are assistant tool-call arguments that read a recognized SKILL.md; recommendations/use-references are assistant-authored text such as "Using the x skill" or "Recommended next skill: /x". Reproduce this snapshot with node scripts/analyze-skill-usage-actors.mjs --cutoff 2026-07-04T20:45:12.007Z; the cutoff prevents this analysis session from changing its own counts during later verification.

1b. User-Invoked vs. Agent-Invoked Skill Counts

The actor split changes the interpretation of "unused." Direct user commands cover 91 of 184 active skills, but assistant-side skill-contract loads touch a much broader slice of the registry, especially orchestration and design-tree skills that are often loaded by the agent during execution rather than typed by the user.

Interpretation: "Agent invoked" is not identical to "the user typed a skill." The strongest agent-side signal is a contract-load event because it proves the assistant read a skill contract while acting. The session-skill-pair count is less inflated than raw reads and is the best single count for "agents invoked skills." Recommendation events are reported separately because they may be handoff text rather than actual execution.

2. Top Skills by Invocation (all-time, Claude + Codex)

3. Usage Concentration — Head vs. Dead Tail

The distribution is still skewed, but the actor split matters. For direct user commands, the top 3 (ship, ship-end, investigate) account for 2,139 of 3,215 invocations (66.5%). For assistant contract loads, the top 5 are research/design orchestrators (customer-discovery, user-flow-map, ux-variations, competitive-analysis, ui-interview), which shows agents exercise a different part of the skill surface.

Caveat — "unused" ≠ "dead": the 93-skill tail is measured only as never directly typed by the user. Rich transcript parsing now proves agents do load skill contracts independently, so direct-command absence is not enough to delete a skill. Separating dead from new/library/agent-only/parent-invoked is exactly what a skill-inventory pass would resolve (Rec 2).

4. Manual Prompts That Shadow Existing Skills

Your follow-up question: cases where I'm manually typing prompts that skills already cover. I scanned all 15,349 manual (non-command) prompts for intents that map to a shipped skill. The finding: the gap is discovery/habit, not missing skills — the skill exists, and the request is still hand-typed. Counts are exact pattern matches (with real examples pulled from history).

4.1 /investigate — the standout gap (~3:1 manual-to-skill)

Every bare "why is X" skips the skill's claim-validation → root-cause → git-history-trace and just gets an ad-hoc answer. This is the largest single shadowed intent and the one where skipping the skill costs the most.

4.2 Git / docs / roadmap shipping asks typed as sentences

Why this is a habit gap, not a skill gap: /ship is muscle memory (1,598 uses) because it's the natural end-of-turn action. The shadowed skills fire from a mid-work realization ("wait, why is this...") where you're already typing a sentence, so the slash prefix never happens. Bare "why is X" also doesn't feel like it needs a skill — but that's exactly /investigate's job.

Count precision: "commit & push" is the tightened count (requires git/github/commit-message/branch context, excludes database/seed pushes). A looser "commit OR push" match returns ~696 but folds in DB-push false positives, so the tightened 211 is the conservative figure. Buckets use first-match keyword classification and are not mutually exhaustive.

5. Claude vs. Codex Personality Split

The two tools are used for different jobs — usage rates reflect that clearly:

• Claude = the shipping surface. Dominated by ship / ship-end / investigate / sync — day-to-day dev flow.
• Codex = the skill-development bench. Its top direct commands are benchmark-test-skill (82), session-triage (65), benchmark-agent-review (32), and customer-discovery (32) — authoring, testing, and research/design orchestration rather than routine product shipping.
• Only 708 of 9,267 Codex prompts (7.6%) are command prompts, and 501 are active repo-skill invocations. Claude has 2,714 direct repo-skill invocations across 11,704 prompts (23.2%), so Claude remains the heavier direct-command surface.

6. Token & Cost Check

7. Ranked Recommendations

8. Highest-Impact View (avoided manual prompts)

1. /investigate prompt-nudge hook — ~550 historical hits, the largest single shadowed intent, and the highest-value one (it restores claim-validation before an answer).
2. skill-inventory audit — resolves the 93-skill direct-command tail ambiguity in one pass; a smaller and better-labeled registry improves skill recall for every future session.
3. ship-end / docs nudge — ~470 combined commit-push/update-docs sentences; pure convention change, zero new code.
4. Already-won baseline: /ship at 1,598 uses is the single largest user-invoked automation in the dataset — proof the head of the distribution is healthy; the opportunity is in the tail, the agent-only surface, and the shadowed mid-work intents.

9. Evidence Matrix

10. Confidence & Assumption Register

Alternatives considered

• Do nothing — the head is healthy. Rejected as a full answer: true for shipping, but it leaves the ~550 shadowed /investigate asks (the highest-value gap) on the table.
• Auto-run /investigate on any "why" prompt. Rejected — too aggressive; a suggestion nudge preserves user control and avoids false triggers on rhetorical questions.
• Delete all 93 direct-command-unused skills now. Rejected — can't distinguish dead from new/library/agent-only/indirect without the inventory audit first (Rec 2).

Source coverage gaps

• Claude history.jsonl has no token/cost metadata; all dollar costs unavailable.
• Agent-side skill contract loads are counted, but semantic categories such as parent-invoked, framework-invoked, library-only, and genuinely dead still need inventory classification.
• Codex rollup token events were not parsed for this update; rich Codex records were used only for assistant messages/tool-call arguments.
• Intent buckets use first-match keyword classification and are not mutually exhaustive.

Downstream implications

• Approving Rec 1 routes to update-config (a settings.json hook edit — local, reversible).
• Approving Rec 2 routes to /skill-inventory (available in current session) to classify the 93-skill direct-command tail.
• Approving Rec 3 edits CLAUDE.md shared conventions, which propagates to future sessions repo-wide.

11. Glossary Additions Gate

Proposed for research/glossary.md (flat repo — single parent glossary). Only approved terms are appended during the confirmed-page write step. Note: "manual ask" / "steering prompt" may already exist from a prior analyze-sessions run — if so, these Approve actions are no-ops.

12. Artifact Destination & Proposed File Changes

Already written this run (review-stage): this alignment page; the machine-local insights store update (.session-insights/insights.md + watermark — gitignored, not committed); and the prompt-history log under prompts/analyze-sessions/.

On approval (Stage 3), the proposed canonical writes are:

• Write the approved report to research/analyze-sessions-skill-usage-rates.md (flat research path, matching existing layout).
• Append approved glossary terms to research/glossary.md (create with standard header if missing).
• Convert this page to confirmed and update alignment/index.html.
• Commit + push the page, report, glossary, and index (the insights store stays local/gitignored).

13. Post-Approval Route

If Rec 1 is approved, the natural follow-up is update-config to wire the /investigate prompt-nudge hook. If Rec 2 is approved, route to /skill-inventory (skill-dev pack, installed ✓). Both are available in the active session.

14. Compile