Benchmark Agent Review: provision-agentic-config
100%
Claude hard pass rate, 3/3 evaluated
100%
Codex hard pass rate, 3/3 evaluated
86.5
Median subjective score
84-89
Subjective score range
Verdict
Good. The retained AGENTS.md outputs are useful and mostly ergonomic. They include orchestration, verification, task management, direct-to-primary shipping, monorepo safety, no-GitHub-Actions guidance, and the correct $exec handoff.
Not excellent. Artifact-reference ergonomics are consistently weak. Some handoffs name only AGENTS.md; Codex runs sometimes surface benchmark temp paths that are not useful outside the sandbox.
Score Table
| Runner | Runs | Subjective Scores | Grade | Primary Note |
|---|---|---|---|---|
| Claude | 0, 1, 2 | 88, 86, 84 | Good | Strong policy content; artifact references are generic. |
| Codex | 0, 1, 2 | 89, 84, 87 | Good | Correct route and policy blocks; temp artifact paths weaken handoff quality. |
Strengths
- Each run clearly provisions
AGENTS.mdfromworkflow.md. - The policy sections are concrete enough for a next operator to follow.
- All outputs avoid GitHub Actions drift and end with
Recommended next command: $exec.
Remediation
| Finding | Owner | Change | Proof |
|---|---|---|---|
| Handoffs do not consistently identify the artifact with a repo-relative path. | global/codex/provision-agentic-config/SKILL.md and Claude mirror |
Require ./AGENTS.md or ./CLAUDE.md in final output and reject temp fixture paths. |
Focused benchmark/rubric check for repo-relative artifact references and no /private/var or /tmp links. |
| Generated artifacts do not always include a concise source/verification note. | Provisioning skill artifact contract | Add a small source and verification note naming workflow.md and benchmark coverage validation when present. |
Focused rerun confirms retained AGENTS.md includes source and verification evidence. |
| The deterministic artifact-reference rubric could distinguish missing paths from temp paths. | tests/layer4/setups/tier23-global-workflows.setup.ts |
Score repo-relative paths highest, temp paths separately, and missing artifact references lowest. | pnpm --dir tests exec vitest run --project layer1 bench-setups -- -t provision-agentic-config |
Next Work
$targeted-skill-builder provision-agentic-config repo-relative artifact handoff