Devtool Docs Audit - agentic-skills

Executive Verdict

P0 None. The current docs are not missing the basic install, pack, restart, or mode concepts needed by an experienced shell user.

P1 The activation path is still too implicit. A reader can learn commands, but the docs still do not give a single first-success route from clone to visible proof.

P1 Troubleshooting remains fragmented. Restart, refresh, mode, approval-packet, drift, and validation recovery are documented, but not in one symptom-led support path.

P1 Proof/example artifacts are strong but under-used. tasks/history.md, benchmark outputs, Skills Showcase generated proof data, and devtool research artifacts demonstrate real usage, but README does not present them as the adoption proof path.

P2 Script references are better than before, but still split across multiple docs. The most common commands are covered; exact "which script answers which question" guidance is still scattered.

P2 Team rollout guidance is still research-only. Public docs cover .agents/project.json and refresh behavior, but not a concise team checklist.

P2 Stale install.sh references remain in the devtool research chain. README and skills reference now correctly use ./init.sh.

Findings

P1 - First-success route is still implicit

Claim: README and workflow docs explain the pieces, but they do not package a fastest proof path.

Evidence: README opens with ./init.sh, lists pack install commands, and explains reload/restart behavior. docs/codex-workflow.md explains $exec and tasks/todo.md. A search for first-success language in README/docs finds no dedicated section.

Inference: users must synthesize the activation sequence themselves.

Decision impact: add a README "First successful cycle" section before larger docs work.

1. Clone this repo and run ./init.sh.
2. In a target project, run scripts/pack.sh install <pack-or-skill>.
3. Restart Claude Code or start a fresh Codex session if the skill list is stale.
4. Run /pack or $pack, or scripts/pack.sh status, to confirm the project designation.
5. If tasks/todo.md is missing, run a planning skill first.
6. Run /exec or $exec and confirm tasks/history.md plus commit/push evidence.

P1 - Troubleshooting is present as fragments

Claim: the repo documents important recovery facts, but a user with a symptom still has to search across several files.

Evidence: README covers reload/restart; docs/packs.md covers doctor, refresh, stale installs, pinned installs, and update mode; docs/operating-modes.md covers approval-packet lifecycle and hybrid degraded paths; no central troubleshooting page exists.

Inference: troubleshooting content is organized by subsystem, not by user failure.

Decision impact: add a symptom-led docs/troubleshooting.md or README subsection.

P1 - Proof artifacts are strong but under-presented

Claim: the repo has credible dogfood proof, but the landing docs do not point readers to a short proof trail.

Evidence: tasks/history.md, research/devtool-*.md, docs/benchmark-results-matrix.md, and docs/skills-showcase/assets/github-proof-data.js all provide inspectable proof signals. README lacks a proof/examples section.

Inference: existing proof can be converted into adoption value with links and honest boundaries.

Decision impact: add a README "Proof and examples" block.

P2 - Script reference is scattered

Claim: common script commands are documented, but spread across README, pack docs, skills reference, operating modes, and test harness docs.

Decision impact: add docs/scripts-reference.md or expand docs/skills-reference.md with a script reference table covering install/global, project packs, mode/handoff, skill hygiene, showcase, and benchmark commands.

P2 - Team adoption checklist is still missing

Claim: team rollout concerns are known, but public docs stop short of a checklist.

Evidence: docs/packs.md explains committed project designation and generated local roots, while research docs identify checkout-path coupling, restart friction, direct-to-primary defaults, and generated-root discipline.

Decision impact: add a practical team checklist to docs/packs.md.

P2 - Existing devtool research has stale installer terminology

Claim: dogfood research artifacts still mention install.sh, but the current repo uses init.sh.

Evidence: install.sh is absent; README and skills reference use ./init.sh; rg "install.sh" research/devtool-*.md still finds references in multiple active research artifacts.

Decision impact: reconcile those artifacts or add dated historical-context notes before promoting them as examples.

Coverage By Audit Area

Evidence Matrix

Assumptions And Confidence

• High confidence: the current public docs use ./init.sh for global initialization.
• High confidence: no central docs/troubleshooting.md or first-success README section exists.
• High confidence: the older audit was stale and needed refresh.
• Medium-high confidence: a script reference page is better than continuing to scatter command semantics.
• Medium confidence: team adoption should standardize checkout path. Teams may choose different onboarding conventions.
• Medium confidence: stale devtool research should be reconciled rather than caveated. If the repo treats research artifacts as historical snapshots, a dated caveat may be enough.

Alternatives Considered

1. Full documentation rewrite: rejected. The docs are mostly accurate; the highest value is navigation and activation packaging.
2. Only update stale install.sh references: rejected as too narrow. It fixes an accuracy issue but misses activation and support gaps.
3. Add a separate website-first onboarding page: deferred. README and docs pages are the current adoption surface.
4. Implement doc fixes immediately in this audit: rejected for this step. The task requested an audit refresh and alignment review before follow-up doc changes.

Recommended Backlog

1. Add README "First successful cycle" and "Proof and examples" sections.
2. Add docs/troubleshooting.md with symptom-led recovery steps.
3. Add docs/scripts-reference.md or a compact script reference section in docs/skills-reference.md.
4. Add a team adoption checklist and moved-checkout recovery note to docs/packs.md.
5. Reconcile stale install.sh references across active research/devtool-*.md artifacts, or add dated historical-context notes before using those artifacts as examples.

Approval Gates