Skip to content

[Feature]: Document a test harness for scripting gate verdicts non-interactively #2594

@doquanghuy

Description

@doquanghuy

Problem Statement

Spec Kit's gate step prompts operators interactively for a
verdict. There's no documented way to script those verdicts in a
test runner, so:

  • Spec Kit's own CI cannot exercise gate behaviour end-to-end —
    changes to gate prompt format, output channel, or switch routing
    on verdict can regress without coverage.
  • Workflows authored on Spec Kit that depend on gate semantics
    cannot be CI-tested without hand-rolling a pexpect driver
    against an undocumented prompt format.

Proposed Solution

A documented way to script gate verdicts during
specify workflow run. Two possible shapes:

A. --gate-script flag

specify workflow run my-pipeline --gate-script verdicts.yaml
schema: speckit.gate-script/v1
verdicts:
  - {gate_id: "review-overview", iteration: 0, verdict: "improve"}
  - {gate_id: "review-overview", iteration: 1, verdict: "approve"}

B. Document the canonical gate-prompt regex (e.g. anchored
markers like SPECKIT_GATE_PROMPT_BEGIN: <id>
SPECKIT_GATE_PROMPT_END) so contributors can write pexpect
drivers against a stable contract.

A is more CI-friendly; B is more flexible for advanced cases.

Component

Specify CLI (initialization, commands)

AI Agent (if applicable)

All agents

Acceptance Criteria

  • Documented contract for scripting gate verdicts (A or B).
  • At least one example test in Spec Kit's suite exercising an
    improveapprove cycle.
  • Default behaviour preserved: gates still prompt
    interactively when no script is provided.

Additional Context

If gate behaviour is intentionally manual-only, please say so and
I'll close.

AI disclosure: drafted with Claude Opus, human-reviewed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions