Doctrine-Aware Self-Improving AI Roadmap

# Doctrine-Aware Self-Improving AI Roadmap

Date: 2026-07-02
Product path: `unthinkable`
Topic: `doctrine-aware-ai`
Status: active architecture roadmap
Primary prototype target: `prototypes/unthinkable/tactical-battle/playable-battle/`

## Purpose

Omega War should build tactical AI as a doctrine-constrained commander system,
not as a larger flat finite-state machine. Runtime behavior should remain
deterministic, serializable, explainable, and bounded by authored faction-era
doctrine.

Offline training and simulation may fine-tune the runtime data that drives that
commander system: utility weights, plan timings, counter-rules, build orders,
difficulty curves, and failure cases. Shipped browser runtime behavior should
consume reviewed, distilled data tables, utility weights, and authored rules
rather than performing opaque online model inference during a match.

## Sources Checked

| Source | Use in this roadmap |
|---|---|
| `research/unthinkable/tactical-vertical-slice-spec.md` | Confirms the first playable battle target, UK 11th Armoured vs Soviet 2nd Guards Tank Army, baseline AI obligations, save/pause requirement, and browser prototype boundary. |
| `research/unthinkable/tactical-battle-experiment-plan.md` | Provides EXP-08 Baseline AI Loop, EXP-10 save/reload, EXP-13 short skirmish, and EXP-14 full battle assembly as the implementation path. |
| `prototypes/unthinkable/tactical-battle/playable-battle/js/spectator-ai.js` | Shows the current commander-AI approach as a declarative FSM with explainable logs, useful as a migration baseline but not sufficient as the long-term AI architecture. |
| `prototypes/unthinkable/tactical-battle/playable-battle/tests/headless-sim.js` | Confirms current expectations for headless safety, deterministic runs, AI log projection, save/reload, and AI-vs-AI spectator coverage. |
| `research/unthinkable/divisions-1945.md` and `research/unthinkable/proposed-divisions-and-decks-roster.md` | Doctrine and roster identity sources for the Soviet 2nd Guards and UK 11th Armoured profiles. |

## Current Gap

The current tactical AI direction is explainable, but too flat. A single
commander FSM can capture scripted pressure and readable logs, yet it does not
separate doctrine, general personality, difficulty, player adaptation, plan
selection, tactical utility, and unit micro. That makes later "self-improving"
behavior risky: it could either become opaque and unfair or collapse into a
generic optimal RTS opponent that no longer looks like its force.

The next AI milestone should preserve the useful parts of the current prototype:

- headless-safe execution,
- deterministic seeded runs,
- compact serialized state,
- visible commander logs,
- honest scouting and prerequisites,
- ability to lose.

The change is to put those behaviors behind explicit doctrine, planner, utility,
and learning-memory contracts before adding richer adaptation.

## Architecture Direction

Runtime AI should be layered:

1. **Doctrine profile.** Faction-era constraints, allowed plans, preferred unit
   categories, phase priorities, escalation rules, acceptable risk, historical
   tendencies, forbidden behaviors, and losing-state fallback rules.
2. **General personality.** Aggressive, defensive, cautious, attritional,
   armor-biased, artillery-biased, recon-biased, or combined-arms-biased
   tendencies that tune choices inside doctrine.
3. **Difficulty profile.** Reaction delay, scouting accuracy, risk tolerance,
   reserve budget, micro precision, recovery competence, and adaptation strength.
4. **Commander planner.** Selects intent-level battle plans such as seize near
   points, contest center, probe flank, defend yard, tech armor, prepare
   artillery, or commit shock push.
5. **Tactical utility layer.** Scores objectives, targets, retreats, artillery,
   reinforcements, counterattacks, command-post defense, and timing windows.
6. **Unit micro layer.** Executes local engagement behavior: movement, cover,
   focus fire, retreat, resupply, artillery/air use, and regrouping.

Doctrine is the boundary, not flavor text. AI may improve within doctrine; it
must not converge into generic optimal RTS cheese.

## Doctrine Baselines

### Soviet 2nd Guards Tank Army

The Soviet AI should prefer mass, echelon pressure, armor exploitation,
artillery preparation, and shock commitment. It should be allowed to accept
higher losses when pressure creates operational advantage, but it should not
throw units away without a plan or use omniscient counters.

Initial preferred plans:

- seize near points with scouts and tank riders,
- pressure the center with massed armor and infantry,
- prepare attacks with Katyusha or heavy artillery,
- exploit exposed flanks with T-34-85 groups,
- commit IS-2 or IS-3 shock only after visible prerequisites,
- fallback to layered defense and artillery if the initial push fails.

Penalized or forbidden behaviors:

- early unsupported IS-3-style shock,
- perfect hidden-vision anti-armor counters,
- unsupported armor trickles when doctrine calls for mass,
- repeated one-lane attacks after the same failure signal.

### UK 11th Armoured Division

The UK AI should prefer combined arms, recon, deliberate armor commitment,
artillery observation, and air support. It should preserve expensive armor more
than the Soviet AI and use observation and support assets to prepare committed
moves.

Initial preferred plans:

- scout and screen before committing Comets,
- contest center through combined infantry, armor, and AT support,
- use AOP/FOO observation to prepare artillery,
- protect command-post surfaces with deliberate reserves,
- use air support as a timed late answer rather than constant pressure,
- counterattack after stabilizing a threatened sector.

Penalized or forbidden behaviors:

- blind armor rushes unsupported by recon,
- static camping when capture pressure demands action,
- air support without scouting/observation context,
- perfect reaction to unseen Soviet concentrations.

## Serializable Runtime Contracts

### `AiCommanderProfile`

The commander profile is authored or selected before the match and saved with
the battle.

Required fields:

| Field | Purpose |
|---|---|
| `side` | `player` or `enemy` runtime side. |
| `forceId` | Force identifier, such as `uk-11th-armoured` or `soviet-2nd-guards-tank-army`. |
| `doctrineId` | Key into the doctrine data table. |
| `personalityId` | General personality overlay. |
| `difficultyId` | Difficulty profile key. |
| `planWeights` | Base commander-plan utility weights after doctrine and personality modifiers. |
| `riskKnobs` | Risk tolerance, reserve budget, loss acceptance, retreat threshold, escalation threshold. |
| `adaptationSettings` | Which adaptation modes are enabled and how strongly they may move weights. |
| `seed` | Deterministic seed or profile-specific seed salt for reproducible choices. |

### `AiIntentState`

`AiIntentState` extends the existing AI log model and is saved in battle
snapshots.

Required fields:

| Field | Purpose |
|---|---|
| `currentPlan` | Active commander plan id and start time. |
| `planReason` | Short explainable reason for the current plan. |
| `targetObjectives` | Capture points, enemy posts, lanes, or map regions currently prioritized. |
| `buildStep` | Current command-post or deployment step with prerequisite state. |
| `groupAssignments` | Runtime unit ids grouped into probe, main effort, reserve, defense, support, or recovery roles. |
| `knownPlayerHabits` | Compact inferred player-habit signals visible to this AI. |
| `adaptationMemory` | Compact explainable signals and their confidence/cooldown values. |
| `lastAdaptationReason` | Last emitted reason for a plan or weight change. |
| `cooldowns` | Per-plan, per-target, or per-response cooldowns that prevent robotic repetition. |
| `log` | Append-only explainable AI decisions, newest-first or oldest-first by local convention. |

## Doctrine Data Table

Each `AiDoctrineProfile` should include:

- allowed plans,
- preferred unit categories,
- phase priorities,
- historical bias weights,
- doctrine-specific risk limits,
- forbidden behaviors,
- penalized behaviors,
- fallback behaviors when losing,
- adaptation boundaries.

Example table shape:

```js
{
  id: "soviet-2nd-guards-1945",
  forceId: "soviet-2nd-guards-tank-army",
  allowedPlans: ["seize-near", "center-pressure", "flank-probe", "armor-tech", "artillery-prep", "shock-commit", "fallback-defense"],
  preferredCategories: { ARMOR: 1.35, INF: 1.05, ARTY: 1.2, REC: 0.9, AIR: 0.75 },
  phasePriorities: { A: ["seize-near", "center-pressure"], B: ["armor-tech", "artillery-prep"], C: ["shock-commit"] },
  forbiddenBehaviors: ["hidden-vision-counter", "early-wonder", "generic-cheese"],
  fallbackPlans: ["fallback-defense", "artillery-prep", "counterattack-center"]
}
```

## Learning Memory And Training Signals

Runtime learning memory stores only compact, explainable signals. It does not
store opaque model state or player-identifying data.

Initial signals:

- player overuses center rush,
- player ignores north flank,
- player ignores south flank,
- player delays anti-armor,
- player punishes unsupported armor,
- player overcommits to artillery targets,
- AI lost too many units during probes,
- artillery fired with poor effect,
- command-post defense arrived too late,
- reserve budget was too low after Phase B.

Each signal should carry count, confidence, last observed time, decay/cooldown,
and a short evidence note suitable for logs.

Every adaptive change must emit a log entry, for example:

- `General shifted from center pressure to south probe because player overcommitted to CP-C in 3 prior engagements.`
- `Doctrine retained armor-first escalation; difficulty lowered reaction delay but did not grant hidden vision.`

AI decision and action logs are also offline-training inputs. The runtime should
record enough structured context for later simulation and fine-tuning work:
selected plan, candidate plans considered, utility scores or rank buckets, issued
orders, observed prerequisites, scout-visible enemy facts, outcome summaries, and
the doctrine rule or difficulty knob that bounded the decision. Those logs may
contribute to offline training datasets or simulation replay analysis, but any
resulting weight changes must still be reviewed and distilled back into doctrine
tables, utility weights, or authored rules before shipping.

## Self-Improvement Modes

### Mode 1: Doctrine-Bounded Tuning

The AI adjusts weights, timing, plan selection, retreat thresholds, reserve
budgets, and support timing within hard doctrine limits. This is the first
runtime-safe mode.

Acceptance:

- adaptation remains serializable,
- logs explain every significant shift,
- forbidden doctrine behaviors remain blocked,
- repeated play feels less exploitable without becoming generic.

### Mode 2: Player-Specific Adaptation

The AI identifies repeated player habits inside a short match or across an
approved local profile and selects doctrinal counters. The counter must still be
earned through scouting and visible prerequisites.

Acceptance:

- player habits are compact labels, not opaque player models,
- counters have cooldowns and anti-oppression caps,
- difficulty controls how quickly and strongly the AI adapts,
- the player can still surprise or break the read.

### Mode 3: Offline Open-Ended Exploration

Offline training and simulation may search for candidate behaviors, build
orders, plan weights, and counter-rules. Runtime AI decision/action logs may feed
those offline datasets when they are structured, reviewable, and stripped of
player-identifying data. Nothing opaque ships directly into the browser runtime.
Outputs must be reviewed and distilled into doctrine tables, utility weights, or
authored rules.

Acceptance:

- offline outputs and source logs are inspectable,
- runtime consumes distilled data and utility weights only,
- doctrine constraints are applied before shipping,
- experiments identify gaps without turning live matches into opaque online
  inference.

## Difficulty And Fun Rules

Difficulty tunes competence, not cheating.

| Difficulty | Behavior contract |
|---|---|
| Easy | Slower reactions, conservative attacks, weaker adaptation memory, more readable telegraphs, forgiving retreat and reserve behavior. |
| Normal | Doctrine-faithful plan switching, fair scouting, moderate retreats, basic counter-adaptation, honest command-post and phase gating. |
| Hard | Better timing, stronger grouping, better target priority, faster recovery, richer player-habit counters, but still no impossible economy or unbounded hidden vision. |

No difficulty level should grant unbounded hidden vision, impossible economy,
instant perfect counters, invisible late powers, or hidden prerequisite skips.

Anti-oppression guardrails:

- cap losing-streak pressure,
- preserve comeback windows,
- rotate counters instead of repeating a perfect answer,
- keep telegraphs readable,
- keep recovery competence below omniscience,
- let the AI make doctrine-plausible mistakes.

## Implementation Path

### Phase 0: Contract Amendment

Amend EXP-08 so the baseline AI loop becomes the first doctrine-aware commander
contract rather than a one-off FSM. Keep implementation small: one Soviet
opponent profile first.

Required outputs:

- `AiCommanderProfile` shape,
- `AiDoctrineProfile` data table for Soviet 2nd Guards,
- `AiIntentState` extension to the current AI log model,
- log entries for plan selection and adaptation reasons.

### Phase 1: Soviet Opponent Upgrade

Upgrade the current Soviet opponent first. Keep the old FSM behavior as a
baseline for comparison until the doctrine planner passes the same headless
tests.

Required behaviors:

- captures near points,
- contests center,
- escalates Phase B/C honestly,
- defends command surfaces,
- retreats damaged units sometimes,
- preserves reserves according to difficulty,
- adapts to repeated center rush,
- does not violate Soviet doctrine constraints.

### Phase 2: UK Spectator AI Reuse

Reuse the same framework for the UK spectator AI, but with UK 11th Armoured
doctrine and a different personality profile.

Required behaviors:

- recon-first opening,
- deliberate armor commitment,
- combined-arms center contest,
- artillery observation bias,
- air-support timing,
- stabilizing defense before counterattack.

### Phase 3: Offline Tuning Harness

Run AI-vs-script and AI-vs-AI simulations to tune weights and identify brittle
plans. Include structured AI decision/action logs in offline replay analysis
where useful. Distill results into data and authored rules before shipping.

Required outputs:

- scenario scripts,
- run summaries,
- decision/action log schema and replay importer,
- candidate weight diffs,
- rejected behaviors with doctrine/fun reasons,
- updated doctrine tables or utility weights.

## Test Plan

Create AI-vs-script and AI-vs-AI scenario tests for:

- captures near points,
- contests center,
- escalates Phase B/C honestly,
- retreats damaged units,
- defends command surfaces,
- adapts to repeated center rush,
- adapts to repeated flank play,
- does not violate doctrine constraints.

Run three adaptation experiments:

- doctrine-bounded only,
- player-specific adaptation,
- offline open-ended tuning distilled into data.

Validate general personalities:

- aggressive wins or loses by pressure,
- defensive stabilizes and counterattacks,
- cautious preserves units,
- artillery-biased prepares attacks before committing.

## Acceptance Criteria

- AI feels less exploitable after repeated play.
- AI decisions remain explainable in logs.
- AI still looks like its faction and commander.
- AI can lose without collapsing into one-lane stupidity.
- Difficulty increases competence without hidden omniscience or impossible
  economy.
- Offline-trained runtime weights are allowed after review, but live-match AI
  remains deterministic, inspectable, and doctrine-bounded.
- Save/reload preserves commander profile, intent state, adaptation memory,
  cooldowns, and log context.

## Integration With Existing Experiments

| Existing experiment | Required AI-roadmap implication |
|---|---|
| EXP-08 Baseline AI Loop | Upgrade scope to doctrine-aware commander contract for the Soviet opponent. |
| EXP-10 Save/Pause/Reload | Include commander profile, intent state, adaptation memory, and cooldowns in battle snapshots. |
| EXP-11 Performance Ceiling Probe | Track AI think cost, group-assignment cost, and utility-evaluation count. |
| EXP-13 First Compiled Skirmish Thread | Use the doctrine-aware Soviet profile if it is accepted; otherwise keep the old FSM as fallback and log the gap. |
| EXP-14 Full Prototype Assembly | Validate difficulty, adaptation, doctrine constraints, and anti-oppression guardrails in the moderated-test candidate. |

## Active Deferrals

- Opaque online model inference during live browser matches.
- Cloud or account-level player modeling.
- Strategic campaign AI.
- Multiplayer AI fairness policy.
- Full production difficulty balance.
- Automatic shipping of offline-discovered behavior without human review.