Overview Stats
| Source | Messages | Sessions | Date Range |
|---|---|---|---|
| Claude history | 9,549 | 2,826 | 2025-12-13 to 2026-05-25 |
| Codex prompt history | 1,837 | 564 | 2026-01-08 to 2026-05-25 |
Top Projects By Message Volume
| Count | Project |
|---|---|
| 1,322 | /home/georgeqle/projects/experiments/metternich-engine |
| 1,271 | /home/georgeqle/projects/apps/bismarck-ai/v0.3/AWS |
| 694 | /home/georgeqle/projects/poke/monorepo |
| 694 | /home/georgeqle/projects/apps/bismarck-ai/v0.3/monorepo |
| 670 | /home/georgeqle/projects/web/dev/bismarck-ai/bismarck-v0.4 |
| 347 | /home/georgeqle/projects/tools/dev/claude-skills |
Method note: compact Claude and Codex prompt histories were parsed fully. Codex rich session files were scanned for metadata enrichment. Pattern counts are regex-based and intentionally conservative where possible, but broad terms like "research" and "should" include some non-skill project messages.
Claims
Claim 1: Alignment pages are improving understanding when they carry decisions inline.
Evidence includes explicit positive feedback about multiple-choice answers in HTML pages and follow-up behavior where compiled YAML answers are pasted back into agents. The strongest signal is not aesthetic; it is that the user can review the page, answer gates, and use those answers to steer later skill output.
Evidence-backedHigh confidence
Claim 2: The biggest remaining problem is output substance, not the existence of HTML.
The clearest correction says prior skills gave thin summaries and lost context. Later corrections target research freshness, date context, evidence quality, and translation of research into the HTML page.
Evidence-backedHigh confidence
Claim 3: Cross-skill routing and final handoffs remain a recurring misunderstanding source.
There are repeated prompts about recommended next steps, wrong competitive-analysis routing, AFPS route correctness, and whether docs still match current skill setup. This affects multiple skills, especially research and workflow skills.
Evidence-backedHigh confidence
Claim 4: Alignment pages should be role-filtered, not universal.
The history supports durable research/planning/spec/report skills producing alignment pages, but also includes explicit corrections that operational skills such as provision-agentic-config and scaffold may not need them. This is a scope-control issue across all skills.
Evidence plus inferenceMedium-high confidence
Evidence Matrix
| Major Claim | Source Evidence | Inference | Confidence | Assumption Status | Decision Impact |
|---|---|---|---|---|---|
| Alignment pages improve understanding when gates are inline and answerable. | Claude history line 9436, 2026-05-23: user says they enjoy multiple-choice answers in HTML alignment pages, but wants questions next to relevant sections. | Positive feedback attaches to the review mechanics, while the requested change points to spatial/contextual alignment as the key driver. | High | Evidence-backed | Keep HTML gates, but require inline placement near governed content. |
| Thin summaries and context loss are the main failure mode. | Claude history line 9392, 2026-05-22: user reports skills produced thin summaries and lost context, and asks for all proposed Markdown content in HTML. | The page is only useful if it preserves the whole decision context, not a polished summary. | High | Evidence-backed | Audit skills for no-context-loss, not just page creation. |
| Research skills need stronger source recency and time context. | Claude history line 9464, 2026-05-23: user cites stale data and unclear timeframes for stats such as "4.5B spend mid-2025". | Research HTML should expose source date, event date, metric period, and uncertainty for every material stat. | High | Evidence-backed | Make research freshness/date metadata a gate and benchmark criterion. |
| Final routing is still brittle across skills. | Claude history line 8921, 2026-04-12 asks research skills on Codex to explain recommended next steps; line 9526 reports competitive-analysis page approval notes saying next-step routing is wrong and should hand off to journey-map. | Users need final output to identify the next skill and why it follows from the workflow, using current runner syntax. | High | Evidence-backed | Strengthen route-contract validation for research and workflow skills. |
| Approval friction moved from TUI questions to HTML gates, but the boundary is still unsettled. | Claude history line 8805 says Codex run asks for approval a lot; line 8815 says default should be no gates or approval; line 9502 asks that skills avoid TUI questions and use HTML pages for interrogation. | Execution skills should avoid approval gates by default, while durable research/spec decisions can use HTML gates. | Medium-high | Evidence-backed, policy inference required | Classify each skill by output role before applying gates. |
| Some skills should be exempt from alignment pages. | Claude history line 9461 names provision-agentic-config as unnecessary for alignment pages; line 9466 adds scaffold too. | The universal durable-output rule is too broad unless paired with an explicit exemption/role taxonomy. | Medium-high | Evidence-backed for named skills; inferred for similar operational skills | Create a cross-skill alignment-page applicability matrix. |
Evidence Coverage Gate
Is the evidence coverage sufficient to treat the audit conclusions as actionable across all skills?
Categorized Patterns
| Pattern | Count | Sessions | Real Examples | Interpretation |
|---|---|---|---|---|
| Alignment / HTML / browser review | 338 | 265 | Questions about alignment pages, dark mode, browser open, Compile Answers, YAML, and whether pages improve understanding. | The alignment page has become a core skill-output surface, not an edge feature. |
| Questions, gates, approval | 340 | 306 | Requests to move questions into HTML, put questions near content, avoid TUI AskUserQuestion, and reduce approval gates for run loops. | The user wants decision capture, but only in the right workflow phase and UI surface. |
| Route, handoff, next step | 510 | 385 | Research skills must explain recommended next step; competitive-analysis should route to journey-map; AFPS route questions recur. | Skill outputs often fail when they omit or stale-copy the next route. |
| Research quality and freshness | 142 | 120 | User asks for recent data, date/time context, stronger research quality, and better translation into HTML pages. | Research skills need stronger evidence metadata and caveat handling. |
| Explicit confusion or understanding checks | 119 | 105 | User asks whether workflow understanding is accurate, what commands mean, and whether docs still match skills. | Output should include enough workflow context to prevent the user from reverse-engineering the skill system. |
Skill-Specific Signals
| Skill / Surface | Observed Signal | Output Quality Implication |
|---|---|---|
competitive-analysis | Approval YAML says routing is wrong and should hand off to journey-map. | Alignment pages can reveal bad routing before canonical writes; route contracts need tests. |
icp and research skills | User reports stale data and insufficient time context. | Research pages should include evidence dates, metric periods, confidence, and source gaps. |
run / ship | User reports Codex run asking for too much approval and wants no default gates. | Operational loops should not inherit research approval semantics. |
provision-agentic-config and scaffold | User names them as likely unnecessary alignment-page producers. | Use an applicability matrix; do not apply page generation by file-write alone. |
idea-scope-brief | User questions bootstrap routing as premature and later asks about lane tightness versus breadth. | Outputs should distinguish exploratory breadth from downstream lane ownership. |
analyze-sessions | User asks whether alignment pages improve understanding, then corrects scope to all skills. | Analysis outputs need to make their target scope explicit early. |
Ranked Recommendations
| Rank | Pattern | Frequency Signal | Recommendation Type | Suggested Change | Validation Expectation |
|---|---|---|---|---|---|
| 1 | Alignment pages must preserve decision context. | 338 alignment records; explicit context-loss correction. | Standing contract + benchmark | Require every research/spec/report HTML page to render full proposed deliverables, evidence matrix, assumptions, source gaps, and inline questions. | Layer1 alignment-gates test plus benchmark fixture that evaluates HTML artifacts, not only Markdown. |
| 2 | Route handoffs drift across skills. | 510 route/handoff records; direct competitive-analysis correction. | Skill contract + route audit | Centralize AFPS route expectations and assert final command syntax for Codex/Claude separately. | Focused route-contract tests over business-discovery, customer-lifecycle, research-roadmap, roadmap, feature-interview. |
| 3 | Approval gates are useful in research but harmful in execution loops. | 340 gate/approval records; explicit run approval friction. | Skill taxonomy | Classify skills as research/planning/spec/report/prototype versus execution/shipping/config/scaffold; apply HTML gates only to the first group unless explicitly requested. | Repository-wide exemption test that rejects alignment-page clauses in operational loop skills and checks expected durable-output producers. |
| 4 | Research freshness and metric dates need explicit handling. | 142 research-quality records; explicit stale-data correction. | Research contract | Require source date, event date, metric period, confidence, and source coverage gaps for material research claims. | Research skill benchmark smoke that fails undated stats or stale-source summaries. |
| 5 | User questions should live beside governed content. | Direct positive feedback with placement correction. | HTML UI convention | Place each required question under the evidence/recommendation/file-change section it governs; avoid bottom-only decision sections. | HTML DOM check for question blocks near governed section anchors and no separate generic decisions section. |
Highest-Impact Automations
- Cross-skill alignment applicability matrix: prevents unnecessary HTML pages and missing pages in durable research/spec/report skills.
- Route-contract validator: catches stale next-step handoffs such as competitive-analysis skipping journey-map.
- Research evidence lint: flags stats without date, timeframe, source category, confidence, or caveat.
- HTML artifact benchmark evaluator: scores generated pages for gates, full content, and compile YAML behavior.
- Final-handoff normalizer: enforces active-runner command syntax and suppresses downstream routes during pre-approval stops.
Alternatives Considered
| Alternative | Verdict | Reason |
|---|---|---|
| Keep alignment pages for every skill that writes any file. | Rejected | History includes named exceptions and run/ship approval friction. |
| Remove HTML pages and use chat-only approval. | Rejected | User explicitly likes multiple-choice HTML answers and uses compiled YAML. |
| Only improve visual styling such as dark mode. | Lower confidence / incomplete | Dark mode matters, but later corrections emphasize research quality and context preservation. |
| Make one universal alignment template. | Partial | Shared controls help, but research, spec, report, prototype, and execution skills need different gate semantics. |
Source Coverage Gaps
This audit uses local prompt histories, not assistant responses or generated HTML files from every project. That means it captures user corrections and requests well, but it cannot prove how often agents complied after each correction without a deeper artifact scan. Counts for broad patterns are useful for prioritization, not exact defect rates.
Recommended Path Gate
Which remediation path should be treated as the priority after this audit?
Approval Gates
Assumptions / Confidence Gate
Do the stated confidence levels and source limitations match how you want this audit interpreted?
Scope / Non-Goals Gate
Is the scope correct: all skills, with extra scrutiny on alignment pages, research quality, route handoffs, and approval gates?
Artifact Destination Gate
Should this HTML page remain the review artifact for this audit?
Proposed File Changes Gate
Should downstream file changes be deferred until after this page's compiled YAML is provided?
Post-Approval Route Gate
After approval, what should the next action optimize for?
Compile Answers
Answer every required gate above. The compiled YAML is the approval payload to paste back into the agent.