|
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335 |
- ---
- name: 'step-04-generate-tests'
- description: 'Orchestrate adaptive red-phase test scaffold generation (TDD red phase)'
- nextStepFile: '{skill-root}/steps-c/step-04c-aggregate.md'
- ---
-
- # Step 4: Orchestrate Adaptive Red-Phase Test Scaffold Generation
-
- ## STEP GOAL
-
- Select execution mode deterministically, then generate red-phase API and E2E test scaffolds (TDD RED PHASE) with consistent output contracts across agent-team, subagent, or sequential execution.
-
- ## MANDATORY EXECUTION RULES
-
- - 📖 Read the entire step file before acting
- - ✅ Speak in `{communication_language}`
- - ✅ Resolve execution mode from config (`tea_execution_mode`, `tea_capability_probe`)
- - ✅ Apply fallback rules deterministically when requested mode is unsupported
- - ✅ Generate red-phase test scaffolds only (TDD red phase)
- - ✅ Wait for required worker steps to complete
- - ❌ Do NOT skip capability checks when probing is enabled
- - ❌ Do NOT generate active passing tests (this is red phase)
- - ❌ Do NOT proceed until required worker steps finish
-
- ---
-
- ## EXECUTION PROTOCOLS:
-
- - 🎯 Follow the MANDATORY SEQUENCE exactly
- - 💾 Wait for subagent outputs
- - 📖 Load the next step only when instructed
-
- ## CONTEXT BOUNDARIES:
-
- - Available context: config, acceptance criteria from Step 1, test strategy from Step 3
- - Focus: orchestration only (mode selection + worker dispatch)
- - Limits: do not generate tests directly (delegate to worker steps)
- - Dependencies: Steps 1-3 outputs
-
- ---
-
- ## MANDATORY SEQUENCE
-
- **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
-
- ### 1. Prepare Execution Context
-
- **Generate unique timestamp** for temp file naming:
-
- ```javascript
- const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
- ```
-
- **Prepare input context for both subagents:**
-
- ```javascript
- const parseBooleanFlag = (value, defaultValue = true) => {
- if (typeof value === 'string') {
- const normalized = value.trim().toLowerCase();
- if (['false', '0', 'off', 'no'].includes(normalized)) return false;
- if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
- }
- if (value === undefined || value === null) return defaultValue;
- return Boolean(value);
- };
-
- const subagentContext = {
- story_acceptance_criteria: /* from Step 1 */,
- test_strategy: /* from Step 3 */,
- knowledge_fragments_loaded: /* list of fragments */,
- config: {
- test_framework: config.test_framework,
- use_playwright_utils: config.tea_use_playwright_utils,
- use_pactjs_utils: config.tea_use_pactjs_utils,
- pact_mcp: config.tea_pact_mcp, // "mcp" | "none"
- browser_automation: config.tea_browser_automation,
- execution_mode: config.tea_execution_mode || 'auto', // "auto" | "subagent" | "agent-team" | "sequential"
- capability_probe: parseBooleanFlag(config.tea_capability_probe, true), // supports booleans and "false"/"true" strings
- provider_endpoint_map: /* from Step 1/3 context, if use_pactjs_utils enabled */,
- },
- timestamp: timestamp
- };
- ```
-
- ---
-
- ### 2. Resolve Execution Mode with Capability Probe
-
- ```javascript
- const normalizeUserExecutionMode = (mode) => {
- if (typeof mode !== 'string') return null;
- const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
-
- if (normalized === 'auto') return 'auto';
- if (normalized === 'sequential') return 'sequential';
- if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
- return 'subagent';
- }
- if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
- return 'agent-team';
- }
-
- return null;
- };
-
- const normalizeConfigExecutionMode = (mode) => {
- if (mode === 'subagent') return 'subagent';
- if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
- return mode;
- }
- return null;
- };
-
- // Explicit user instruction in the active run takes priority over config.
- const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
-
- const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(subagentContext.config.execution_mode) || 'auto';
- const probeEnabled = subagentContext.config.capability_probe;
-
- const supports = {
- subagent: runtime.canLaunchSubagents?.() === true,
- agentTeam: runtime.canLaunchAgentTeams?.() === true,
- };
-
- let resolvedMode = requestedMode;
-
- if (requestedMode === 'auto') {
- if (supports.agentTeam) resolvedMode = 'agent-team';
- else if (supports.subagent) resolvedMode = 'subagent';
- else resolvedMode = 'sequential';
- } else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
- resolvedMode = supports.subagent ? 'subagent' : 'sequential';
- } else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
- resolvedMode = 'sequential';
- }
-
- subagentContext.execution = {
- requestedMode,
- resolvedMode,
- probeEnabled,
- supports,
- };
-
- if (!probeEnabled && (requestedMode === 'agent-team' || requestedMode === 'subagent')) {
- const unsupportedRequestedMode =
- (requestedMode === 'agent-team' && !supports.agentTeam) || (requestedMode === 'subagent' && !supports.subagent);
-
- if (unsupportedRequestedMode) {
- subagentContext.execution.error = `Requested execution mode "${requestedMode}" is unavailable because capability probing is disabled.`;
- throw new Error(subagentContext.execution.error);
- }
- }
- ```
-
- Resolution precedence:
-
- 1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
- 2. `tea_execution_mode` from config
- 3. Runtime capability fallback (when probing enabled)
-
- If probing is disabled, honor the requested mode strictly. If that mode cannot be executed at runtime, fail with explicit error instead of silent fallback.
-
- ---
-
- ### 3. Dispatch Worker A: Red-Phase API Test Generation
-
- **Dispatch worker:**
-
- - **Subagent File:** `./step-04a-subagent-api-failing.md`
- - **Output File:** `/tmp/tea-atdd-api-tests-${timestamp}.json`
- - **Context:** Pass `subagentContext`
- - **Execution:**
- - `agent-team` or `subagent`: launch non-blocking
- - `sequential`: run blocking and wait before next dispatch
- - **TDD Phase:** RED (scaffold tests with `test.skip()`)
-
- **System Action:**
-
- ```
- 🚀 Launching Subagent A: RED-PHASE API Test Generation
- 📝 Output: /tmp/tea-atdd-api-tests-${timestamp}.json
- ⚙️ Mode: ${resolvedMode}
- 🔴 TDD Phase: RED (tests emitted as `test.skip()` scaffolds)
- ⏳ Status: Running...
- ```
-
- ---
-
- ### 4. Dispatch Worker B: Red-Phase E2E Test Generation
-
- **Dispatch worker:**
-
- - **Subagent File:** `./step-04b-subagent-e2e-failing.md`
- - **Output File:** `/tmp/tea-atdd-e2e-tests-${timestamp}.json`
- - **Context:** Pass `subagentContext`
- - **Execution:**
- - `agent-team` or `subagent`: launch non-blocking
- - `sequential`: run blocking and wait before next dispatch
- - **TDD Phase:** RED (scaffold tests with `test.skip()`)
-
- **System Action:**
-
- ```
- 🚀 Launching Subagent B: RED-PHASE E2E Test Generation
- 📝 Output: /tmp/tea-atdd-e2e-tests-${timestamp}.json
- ⚙️ Mode: ${resolvedMode}
- 🔴 TDD Phase: RED (tests emitted as `test.skip()` scaffolds)
- ⏳ Status: Running...
- ```
-
- ---
-
- ### 5. Wait for Required Worker Completion
-
- **If `resolvedMode` is `agent-team` or `subagent`:**
-
- ```
- ⏳ Waiting for subagents to complete...
- ├── Subagent A (API RED): Running... ⟳
- └── Subagent B (E2E RED): Running... ⟳
-
- [... time passes ...]
-
- ├── Subagent A (API RED): Complete ✅
- └── Subagent B (E2E RED): Complete ✅
-
- ✅ All subagents completed successfully!
- ```
-
- **If `resolvedMode` is `sequential`:**
-
- ```
- ✅ Sequential mode: each worker already completed during dispatch.
- ```
-
- **Verify both outputs exist:**
-
- ```javascript
- const apiOutputExists = fs.existsSync(`/tmp/tea-atdd-api-tests-${timestamp}.json`);
- const e2eOutputExists = fs.existsSync(`/tmp/tea-atdd-e2e-tests-${timestamp}.json`);
-
- if (!apiOutputExists || !e2eOutputExists) {
- throw new Error('One or both subagent outputs missing!');
- }
- ```
-
- ---
-
- ### 6. TDD Red Phase Report
-
- **Display TDD status:**
-
- ```
- 🔴 TDD RED PHASE: Test Scaffolds Generated
-
- ✅ Both subagents completed:
- - API Tests: Generated with test.skip()
- - E2E Tests: Generated with test.skip()
-
- 📋 All tests assert EXPECTED behavior
- 📋 Activated tests will FAIL until feature is implemented
- 📋 Scaffolds stay skipped until a developer activates the current task
- 📋 This is INTENTIONAL (TDD red phase)
-
- Next: Aggregation will verify TDD compliance
- ```
-
- ---
-
- ### 7. Execution Report
-
- **Display performance metrics:**
-
- ```
- 🚀 Performance Report:
- - Execution Mode: {resolvedMode}
- - API Test Generation: ~X minutes
- - E2E Test Generation: ~Y minutes
- - Total Elapsed: ~mode-dependent
- - Parallel Gain: ~50% faster when mode is subagent/agent-team
- ```
-
- ---
-
- ### 8. Proceed to Aggregation
-
- **Load aggregation step:**
- Load next step: `{nextStepFile}`
-
- The aggregation step (4C) will:
-
- - Read both subagent outputs
- - Verify TDD red phase compliance (all tests have test.skip())
- - Write all test files to disk
- - Generate ATDD checklist
- - Calculate summary statistics
-
- ---
-
- ## EXIT CONDITION
-
- Proceed to Step 4C (Aggregation) when:
-
- - ✅ Subagent A (API red-phase tests) completed successfully
- - ✅ Subagent B (E2E red-phase tests) completed successfully
- - ✅ Both output files exist and are valid JSON
- - ✅ TDD red phase status reported
-
- **Do NOT proceed if:**
-
- - ❌ One or both subagents failed
- - ❌ Output files missing or corrupted
- - ❌ Subagent generated active passing tests (wrong - must be red-phase scaffolds)
-
- ---
-
- ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
-
- ### ✅ SUCCESS:
-
- - Both subagents launched successfully
- - Both worker steps completed without errors
- - Output files generated and valid
- - Tests generated with test.skip() (TDD red phase)
- - Fallback behavior respected configuration and capability probe rules
-
- ### ❌ SYSTEM FAILURE:
-
- - Failed to launch subagents
- - One or both subagents failed
- - Output files missing or invalid
- - Tests generated without test.skip() (wrong phase)
- - Unsupported requested mode with probing disabled
-
- **Master Rule:** TDD RED PHASE requires acceptance test scaffolds marked with `test.skip()`. Mode selection changes orchestration, never red-phase requirements.
|