Analysis date: 2026-05-27 (updated) · Original: 2026-05-26 · Source: ~/.claude/history.jsonl (9,684 records), ~/.codex/history.jsonl (1,879 records) · Date range: 2025-12-13 to 2026-05-27
| Project | Messages | Research Sessions | Phases Completed | Span |
|---|---|---|---|---|
web/dev/b4 | 34 | 13 | 4/4 | Mar 20 – Apr 13 (24d) |
web/dev/loadoutworks | 165 | 11 | 4/4 | Mar 18 – Apr 5 (18d) |
web/dev/calcllm | 33 | 10 | 4/4 | Mar 27 – May 26 (60d) |
web/dev/bismarck-ai/v0.4 | 23 | 8 | 4/4 | Mar 14 – Mar 28 (14d) |
poke/dev/poke-productivity-suite | 17 | 6 | 4/4 | Mar 12 – Apr 4 (23d) |
engines/metternich | varies | 5 | 4/4 | Mar 28 – Apr 4 (7d) |
web/dev/gblock-party-redux | varies | 3 | 3/4 no strategy | May 23 – May 25 (2d) |
web/dev/rewalkthrough | varies | 4 | 3/4 no competitive | May 6 – May 10 (4d) |
games/dev/trail-break | varies | 2 | 2/4 | May 24 – May 25 (1d) |
| Keyword | Claude | Codex | Bias |
|---|---|---|---|
| spec | 771 | 141 | Claude |
| product | 133 | 35 | Claude |
| interview | 66 | 23 | Claude |
| icp | 65 | 18 | Claude |
| research | 47 | 32 | Mixed |
| competitive | 21 | 11 | Mixed |
| gtm | 6 | 7 | Mixed |
| monetization | 12 | 8 | Mixed |
gtm and monetization keywords show roughly even distribution, suggesting growth-phase work may naturally migrate to Codex for autonomous execution.
The core finding: product research workflows split across multiple sessions, multiple days, and multiple packs with no durable state or progress tracker between sessions. Each session starts fresh, frequently repeating work or losing context from prior sessions.
Projects average 5–13 research sessions to reach full phase coverage. The 6 projects that completed all 4 phases needed:
| Project | Sessions | Calendar Days | Sessions/Phase |
|---|---|---|---|
| web/dev/b4 | 13 | 24 | 3.25 |
| web/dev/loadoutworks | 11 | 18 | 2.75 |
| web/dev/calcllm | 10 | 60 | 2.50 |
| web/dev/bismarck-ai/v0.4 | 8 | 14 | 2.00 |
| poke/dev/poke-productivity-suite | 6 | 23 | 1.50 |
| engines/metternich | 5 | 7 | 1.25 |
sessionId and project. Each session boundary means context loss.8 observed regressions where later-phase work dropped back to earlier phases:
The heaviest transition pattern is build ↔ research, not research → research:
Research and building constantly interleave. The user does not complete a full research pipeline before building — they interleave research as it becomes relevant during implementation.
/ship is the #1 command at 848 invocations; research skills appear interspersed between shipping cycles./plan-interview → /plan-interview: 7x in same session (retry pattern)/idea-scope-brief → /idea-scope-brief: 3x/spec-drift → /spec-drift: 3x| Count | Sequence | Assessment |
|---|---|---|
| 7 | /plan-interview → /plan-interview | Retry First run unsatisfactory |
| 3 | /icp → /competitive-analysis | Golden path Intended flow |
| 3 | /spec-drift → /spec-drift | Retry |
| 3 | /idea-scope-brief → /idea-scope-brief | Retry |
| 2 | /positioning → /gtm | Golden path |
| 1 | /icp → /plan-interview → /roadmap | Golden path |
| 1 | /journey-map → /gtm → /monetization | Golden path |
User runs /idea-scope-brief, gets told to install business-discovery pack, does /pack install, then re-runs the next skill in a new session because /clear was used or context was lost.
gblock-party-redux (May 23): /idea-scope-brief → /idea-scope-brief → /idea-scope-brief → /pack → /clear. Three idea-scope-brief retries and a pack install in one session, then ICP in a separate session two days later.Skills produce alignment pages that require user review and YAML compilation. The user reviews asynchronously, sometimes hours or days later, in a new session. The new session has no memory of what was approved or what the next step should be.
calcllm: idea-scope-brief on May 22, ICP on May 23, competitive-analysis on May 24, journey-map on May 25 — each a separate session with a day gap.Research gets interrupted by urgent build/ship work. When user returns to research, context of where they left off is gone. They may re-run a phase or skip ahead without checking what was already completed.
web/dev/b4: 13 research sessions over 24 days, with /ship interleaved 15+ times. Research phases overlap and repeat.The research workflow spans 4 packs: idea-scope-brief (global), business-discovery, customer-lifecycle, business-growth. Each pack must be installed separately. Routing recommendations say "install X pack first" but don't carry forward what work was already done.
| Claim | Source | Evidence | Confidence | Assumption | Decision Impact |
|---|---|---|---|---|---|
| Research workflows are fragmented across 5–13 sessions per project | Claude history.jsonl | Session-project grouping shows 6 projects with 5+ research sessions each | High | None — directly measured | Quantifies the split-path problem. If acceptable, no action needed. |
| Only 30% of projects complete all 4 research phases | Claude history.jsonl | 6 of 20+ projects with research activity reached all 4 phases (discovery, validation, competitive, strategy) | High | Some projects may intentionally skip phases | Determines whether a progress tracker is needed or if incomplete coverage is by design. |
| Phase regressions happen because downstream skills invalidate upstream assumptions | Session sequences + skill SKILL.md routing | 8 observed regressions; ICP and competitive-analysis skills explicitly say "IF downstream impact is Major: prepend /reconcile-research" | Medium | Regressions could also be user-initiated pivots, not skill-driven | If regressions are skill-driven, reconcile-research should be stronger. If user-driven, no fix needed. |
| Build/research interleaving is the dominant workflow, not linear research-then-build | Phase transition counts | build ↔ validation: 62 transitions; build ↔ strategy: 54 transitions. More than research → research transitions. | High | None — directly measured | Any "research pipeline" solution must accommodate interleaved building, not enforce a linear flow. |
| Same-session retries indicate first-run quality issues | Session command sequences | /plan-interview retried 7x; /idea-scope-brief retried 3x; /spec-drift retried 3x | Medium | Retries could be prompt refinement, not quality failure | If quality-driven, skill output needs improvement. If prompt-driven, input handling needs improvement. |
| Cross-pack routing is manually managed by the user | Codex history (May 25) | "we need to make sure that competitive analysis routes to recommend running: pack install customer-lifecycle, then journey-map" | High | None — explicit user statement | Skills must auto-detect installed packs and route accordingly, or a meta-orchestrator is needed. |
| Research execution is Claude-biased; Codex handles meta-workflow | Keyword analysis across both histories | Claude: 1,398 research messages; Codex: 311. All major research keywords show Claude bias except gtm/monetization. | High | Codex history may be less complete (fewer records overall) | Automation investments should target Claude-side skill quality first. |
| Pattern | Frequency | Type | Recommendation | Avoided Manual Prompts |
|---|---|---|---|---|
| Multi-session research fragmentation | 20+ projects | Standing instruction / Convention | Research Progress Manifest: Each research skill writes a research/.progress.yaml file recording completed phases, pending phases, last skill run, and next recommended skill. Skills read this on startup to know where the project left off. |
~40 "where was I" prompts across projects |
| Cross-pack routing gaps | Every project using 2+ business packs | Skill enhancement | Auto-detect installed packs in routing: Business-discovery skills should read .agents/project.json enabled_packs and only recommend installed skills. If the next skill needs an uninstalled pack, auto-prompt /pack install instead of silently recommending a broken route. |
~15 failed route-follows per project |
| Phase regressions (strategy → validation) | 8 observed | Skill enhancement | Reconcile-before-regress gate: When a downstream skill detects upstream-invalidating findings, it should run /reconcile-research automatically before re-routing to the earlier phase, preserving work already done. |
~8 redundant phase re-runs |
| Same-session retries | 13 observed (7+3+3) | Skill enhancement | Retry-aware input handling: When a skill is invoked immediately after itself in the same session, treat the second invocation as a refinement. Read the prior output (alignment page or research file) and ask what to change, rather than starting fresh. | ~13 full re-runs saved |
| Build/research interleaving | 116 transitions | Standing instruction | Accept interleaving as the norm: Do not design research workflows that require completing all research before building. The progress manifest must support "paused at phase X, build in progress, resume research when ready." | N/A (design principle, not automation) |
research/.progress.yaml) — eliminates "where was I" across all projects. ~40 prompts saved.research/.approvals/ so the next session knows what was approved.| Conclusion | Status | What Would Change It |
|---|---|---|
| Research workflows are fragmented | Evidence-backed | Evidence that fragmentation is intentional (e.g., user prefers short sessions) |
| 30% completion rate is a problem | Provisional | Knowing whether incomplete projects were abandoned intentionally or simply never finished research |
| Progress manifest would reduce friction | Provisional | User testing showing whether the manifest is actually read on session start |
| Regressions are skill-driven, not user-driven | Provisional | Examining specific regression sessions to see if the user or the skill initiated the backward step |
| Cross-pack routing is the primary friction source | Evidence-backed | User explicitly stated this in Codex (May 25) |
| Build/research interleaving is the natural workflow | Evidence-backed | Nothing — this is a measured behavioral fact |
Added 2026-05-27 — extends the fragmentation analysis above with evidence on what happens when research skills produce multiple ICP/concept candidates.
The fragmentation analysis (sections 1–8) established that research sessions split across time. This section addresses a different split: research paths branching within a single skill run, producing multiple candidates that downstream skills don't consume.
When /icp produces 2–5 candidates with Value × Accessibility scoring, a primary ICP is selected. The secondary ICPs are written to ## Additional ICPs with condensed sections and a Cross-ICP Analysis (Section 10). But:
/competitive-analysis, /value-prop-canvas, /positioning all run for the primary ICP only.Which skills produce branches, how many, and what format.
| Skill | Version | Multi-path Output | Primary Selection | Secondary Path Fate | Downstream Awareness |
|---|---|---|---|---|---|
idea-scope-brief |
v0.2 | Slugged briefs for pivots | User chooses direction | Written as "related/future concept" | None |
icp |
v0.4 | 2–5 candidates, Value × Accessibility scoring | User-validated at Checkpoint 2 | ## Additional ICPs with condensed sections |
Cross-ICP Analysis only |
competitive-analysis |
v0.6 | Concept-validation mode: Proceed/Pivot/Abandon verdicts | User-validated at Checkpoint 1 | N/A — runs for primary ICP only | None for secondary ICPs |
value-prop-canvas |
v0.2 | Single path only | Primary ICP scoped | N/A | None |
positioning |
v0.3 | Single path only | "One positioning per product" | N/A | None |
What actually happens after /icp produces multiple candidates and a primary is selected.
| Project | Transition Chain After /icp | Secondary ICP Referenced? |
|---|---|---|
content-pyramid | /icp → /ship → /ship-end | No |
calcllm | /icp → /plan-interview → /ship-end | No |
vectorfit | /icp → /ship-end (x2 sessions) | No |
bismarck-v0.4 | /icp → /competitive-analysis → /ship | No |
poke-productivity-suite | /icp → /brainstorm → /plan-interview-ideas | No |
b4 | /icp → /competitive-analysis | No |
metternich | /icp → /brainstorm → /competitive-analysis | Partial — 4 ICPs developed in parallel with identical directory structures |
jit-life-os | /icp → /ship-end → /ship → /icp (re-run) → /competitive-analysis | No |
icp-01-solo-indie, icp-02-hobbyist-learner, icp-03-professional, icp-04-team-lead-director) all developed with identical directory structures, per-ICP persona cards, demo scripts, and objection handling. This was a deliberate multi-ICP product strategy, not the norm.The birth-to-death lifecycle of non-primary ICP/concept candidates.
## Additional ICPs
When idea-scope-brief produces multiple slugged briefs, the lifecycle is similar but shorter:
Three history records show users wanting to branch or revisit without tooling support:
| Claim | Source | Evidence | Inference | Confidence | Assumption | Decision Impact |
|---|---|---|---|---|---|---|
| Secondary ICPs are never consumed by downstream skills | Claude + Codex history; skill SKILL.md files | 0 of 14 post-ICP sessions reference non-primary ICPs. competitive-analysis, value-prop-canvas, positioning all hardcoded to primary ICP scope. | The ICP Cross-ICP Analysis is write-only: it produces comparison data that nothing reads. | High | None — directly measured + confirmed in skill source | Either downstream skills need per-ICP modes, or secondaries should be tracked differently. |
| Deferred ICP candidates are never revisited | Claude + Codex history (11,563 records) | 0 revisitations found. 1 near-miss (calcllm user asked about revisiting but didn't execute). | The current system provides no trigger or prompt to revisit deferred candidates. | High | Users might revisit outside of tracked history (e.g., reading files manually) | If deferred paths matter, a revisit trigger mechanism is needed. If they don't, secondary ICP output should be lighter. |
| idea-scope-brief multi-brief output causes unnecessary branching | Codex history (L1756–1757) | User immediately questioned "Do we need three split paths?" — only 1 observed instance of multi-brief output. | The skill may be over-producing variants; 1 instance is thin evidence but the user reaction was strong. | Medium | One instance may not be representative | idea-scope-brief should default to single brief with explicitly requested variant mode. |
| Users want product-line expansion tools but have none | Claude history (3 projects: calcllm, vectorfit, claude-skills) | 3 separate users/projects expressed desire to expand into adjacent ICPs or product lines, phrased as routing questions. | The gap isn't "run all downstream skills per ICP" but "have a structured way to decide when to expand." | Medium | Expressed desire ≠ actual need; users may have resolved this outside tracked sessions | Progress manifest should include a deferred_paths section with revisit triggers, not full multi-ICP research branches. |
| metternich-engine's 4-ICP parallel development is an outlier, not a pattern to optimize for | Claude history (metternich project) | Only 1 of 7 branching projects developed multiple ICPs in parallel. metternich built 4 full ICP directory structures with persona cards, demo scripts, and objection handling. | This was a deliberate multi-audience product (game engine with solo/hobbyist/pro/team tiers), not a typical ICP branching scenario. | High | None — directly observed | Do not design multi-ICP tooling around the metternich pattern. The common case is 1 primary + deferred secondaries. |
| ICP Cross-ICP Analysis is write-only | ICP SKILL.md (v0.4) + downstream skill source | Section 10 produces Cross-ICP Analysis with value, accessibility, overlap comparisons. No downstream skill reads Section 10 or references it. | Cross-ICP Analysis is effort without return unless converted to actionable revisit triggers. | High | Users might read it manually even though skills don't | Either make Cross-ICP Analysis produce machine-readable revisit triggers, or make it lighter. |
| # | Recommendation | Rationale | Effort | Impact |
|---|---|---|---|---|
| 1 | Add deferred_paths to research/.progress.yamlTrack non-primary ICPs/concepts with: name, reason deferred, revisit trigger (e.g., "if primary ICP acquisition cost exceeds $X" or "if competitive-analysis reveals primary ICP is saturated"), and date parked. |
Lightweight tracking without full research branches. Gives the progress manifest awareness of branching without requiring per-ICP downstream runs. | Low | High |
| 2 | Make ICP Cross-ICP Analysis produce revisit triggers Instead of just comparing candidates, each secondary ICP gets a concrete trigger condition: "Re-evaluate this ICP if [condition]." Written to both the ICP research file and research/.progress.yaml. |
Converts write-only analysis into actionable checkpoints. Downstream skills can check triggers without running full research per-ICP. | Medium | High |
| 3 | Keep downstream skills primary-only by default Add a lightweight "Implications for Secondary ICPs" note (2–3 sentences) in competitive-analysis output, but do NOT run full downstream research per-ICP. |
Running competitive-analysis, positioning, etc. for each ICP creates the "4–8 variation evaluation" problem the user identified. A note is enough to flag relevance without bogging down. | Low | Medium |
| 4 | Do NOT create "research branches" A full branching system (separate research directories per ICP, merge/diff between branches, parallel downstream runs) is over-engineering for the observed 7-project, 0-revisitation pattern. |
Progress manifest + revisit triggers are sufficient. Only metternich developed multiple ICPs in parallel, and that was a deliberate product strategy, not a workflow the tooling should optimize for. | N/A | Avoids over-engineering |
| 5 | idea-scope-brief: explicit convergence in Next Steps When multiple slugged briefs are produced, the Next Steps section must name which is primary and which are deferred, and write deferred briefs to deferred_paths in the progress manifest. |
Prevents the "Do we need three split paths?" reaction by making convergence explicit rather than leaving multiple open paths. | Low | Medium |
deferred_paths Schema
# In research/.progress.yaml
deferred_paths:
- name: "ICP-02: Hobbyist Learner"
source_skill: icp
date_parked: 2026-05-27
reason_deferred: "Lower Value x Accessibility score (3.2 vs primary's 4.1)"
revisit_trigger: "If primary ICP acquisition cost exceeds $50/user OR competitive-analysis shows primary ICP market is saturated"
status: parked # parked | revisiting | promoted | abandoned
- name: "Concept: Team Dashboard Variant"
source_skill: idea-scope-brief
date_parked: 2026-05-27
reason_deferred: "User selected solo-player concept as primary direction"
revisit_trigger: "If solo-player concept hits retention wall at 30-day mark"
status: parked