Nelze vybrat více než 25 témat Téma musí začínat písmenem nebo číslem, může obsahovat pomlčky („-“) a může být dlouhé až 35 znaků.

step-03a-subagent-api.md 12KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271
  1. ---
  2. name: 'step-03a-subagent-api'
  3. description: 'Subagent: Generate API tests only'
  4. subagent: true
  5. outputFile: '/tmp/tea-automate-api-tests-{{timestamp}}.json'
  6. ---
  7. # Subagent 3A: Generate API Tests
  8. ## SUBAGENT CONTEXT
  9. This is an **isolated subagent** running in parallel with E2E test generation.
  10. **What you have from parent workflow:**
  11. - Target features/components identified in Step 2
  12. - Knowledge fragments loaded: api-request, data-factories, api-testing-patterns
  13. - Config: test framework, Playwright Utils enabled/disabled, Pact.js Utils enabled/disabled, Pact MCP mode
  14. - Coverage plan: which API endpoints need testing
  15. **Your task:** Generate API tests ONLY (not E2E, not fixtures, not other test types).
  16. **If `use_pactjs_utils` is enabled AND contract artifacts are explicitly in scope for this subagent:** Apply pactjs-utils conventions from the loaded fragments (`pactjs-utils-overview`, `pactjs-utils-consumer-helpers`, `pactjs-utils-provider-verifier`, `pactjs-utils-request-filter`, `pact-consumer-framework-setup`) when generating contract-level API test scaffolding. Full consumer/provider suite generation — including edits to `vitest.config.pact.ts`, `vitest.config.contract.ts`, `package.json` scripts (`test:pact:consumer`, `test:pact:consumer:run`), and `scripts/publish-pact.sh` — must be triggered explicitly by the parent workflow flag `tea_use_pactjs_utils: true`; do not generate those artifacts implicitly. When in scope, enforce these determinism/FFI-safety rules:
  17. - **Consumer tests**: exactly one `pact.addInteraction()` per `it()` block (use `it.each` for parameterized cases) — PactV4's Rust FFI drops interactions otherwise.
  18. - **Consumer Vitest config**: `vitest.config.pact.ts` must include BOTH `fileParallelism: false` AND `pool: 'forks'` + `poolOptions.forks.singleFork: true`. `fileParallelism: false` prevents workers racing on the shared pact JSON; forks + singleFork prevents the `@pact-foundation/pact` Rust FFI from leaking state across files on Linux CI.
  19. - **Provider Vitest config**: `vitest.config.contract.ts` must include `pool: 'forks'` + `poolOptions.forks.singleFork: true` (same rule as consumer) for multi-file and message-provider suites.
  20. - **Consumer `package.json`**: generate both `test:pact:consumer` (determinism gate calling `scripts/check-pact-determinism.sh`) and `test:pact:consumer:run` (inner vitest invocation).
  21. - **Publish script**: `scripts/publish-pact.sh` normalizes interactions with `jq -S '.interactions |= sort_by(...)'` before `pact-broker publish`.
  22. If `pact_mcp` is `"mcp"`, use SmartBear MCP tools (Fetch Provider States, Generate Pact Tests) to inform test generation.
  23. ---
  24. ## MANDATORY EXECUTION RULES
  25. - 📖 Read this entire subagent file before acting
  26. - ✅ Generate API tests ONLY
  27. - ✅ Output structured JSON to temp file
  28. - ✅ Follow knowledge fragment patterns
  29. - ❌ Do NOT generate E2E tests (that's subagent 3B)
  30. - ❌ Do NOT run tests (that's step 4)
  31. - ❌ Do NOT generate fixtures yet (that's step 3C aggregation)
  32. ---
  33. ## SUBAGENT TASK
  34. ### 1. Identify API Endpoints
  35. From the coverage plan (Step 2 output), identify:
  36. - Which API endpoints need test coverage
  37. - Expected request/response formats
  38. - Authentication requirements
  39. - Error scenarios to test
  40. ### 2. Generate API Test Files
  41. For each API endpoint, create test file in `tests/api/[feature].spec.ts`:
  42. **Test Structure:**
  43. ```typescript
  44. import { test, expect } from '@playwright/test';
  45. // If Playwright Utils enabled:
  46. // import { apiRequest } from '@playwright-utils/api';
  47. test.describe('[Feature] API Tests', () => {
  48. test('[P0] should handle successful [operation]', async ({ request }) => {
  49. // Use apiRequest helper if Playwright Utils enabled
  50. // Otherwise use standard request fixture
  51. const response = await request.post('/api/endpoint', {
  52. data: {
  53. /* test data */
  54. },
  55. });
  56. expect(response.status()).toBe(200);
  57. expect(await response.json()).toMatchObject({
  58. /* expected */
  59. });
  60. });
  61. test('[P1] should handle [error scenario]', async ({ request }) => {
  62. // Test error handling
  63. });
  64. });
  65. ```
  66. **Requirements:**
  67. - ✅ Use `apiRequest()` helper if Playwright Utils enabled (from api-request fragment)
  68. - ✅ Use data factories for test data (from data-factories fragment)
  69. - ✅ Follow API testing patterns (from api-testing-patterns fragment)
  70. - ✅ Include priority tags [P0], [P1], [P2], [P3]
  71. - ✅ Test both happy path and error scenarios
  72. - ✅ Use proper TypeScript types
  73. - ✅ Deterministic assertions (no timing dependencies)
  74. **If Pact.js Utils enabled (from `subagentContext.config.use_pactjs_utils`):**
  75. - ✅ Generate consumer contract tests in `pact/http/consumer/` using `createProviderState({ name, params })` pattern
  76. - ✅ Generate provider verification tests in `pact/http/provider/` using `buildVerifierOptions({ provider, port, includeMainAndDeployed, stateHandlers })` pattern
  77. - ✅ Generate request filter helpers in `pact/http/helpers/` using `createRequestFilter({ tokenGenerator: () => string })`
  78. - ✅ Generate shared state constants in `pact/http/helpers/states.ts`
  79. - ✅ If async/message patterns detected, generate message consumer tests in `pact/message/` using `buildMessageVerifierOptions`
  80. - ✅ **Provider endpoint comment MANDATORY** on every Pact interaction: `// Provider endpoint: <path> -> <METHOD> <route>`
  81. - ⚠️ **Postel's Law for matchers**: Use `like()`, `eachLike()`, `string()`, `integer()` matchers ONLY in `willRespondWith` (responses). Request bodies in `withRequest` MUST use exact values — never wrap request bodies in `like()`. The consumer controls what it sends, so contracts should be strict about request shape.
  82. ### 1.5 Provider Source Scrutiny (CDC Only)
  83. **CRITICAL**: Before generating ANY Pact consumer interaction, perform provider source scrutiny per the **Seven-Point Scrutiny Checklist** defined in `contract-testing.md`. Do NOT generate response matchers from consumer-side types alone — this is the #1 cause of contract verification failures.
  84. The seven points to verify for each interaction:
  85. 1. Response shape
  86. 2. Status codes
  87. 3. Field names
  88. 4. Enum values
  89. 5. Required fields
  90. 6. Data types
  91. 7. Nested structures
  92. **Source priority**: Provider source code is most authoritative. When an OpenAPI/Swagger spec exists (`openapi.yaml`, `openapi.json`, `swagger.json`), use it as a complementary or alternative source — it documents the provider's contract explicitly and can be faster to parse than tracing through handler code. When both exist, cross-reference them; if they disagree, the source code wins. Document the discrepancy in the scrutiny evidence block (e.g., `OpenAPI shows 200 but handler returns 201; using handler behavior`) and flag it in the output JSON `summary` so it is discoverable by downstream consumers or audits.
  93. **Scrutiny Sequence** (for each endpoint in the coverage plan):
  94. 1. **READ provider route handler and/or OpenAPI spec**: Find the handler file from `subagentContext.config.provider_endpoint_map` or by scanning the provider codebase. Also check for OpenAPI/Swagger spec files. Extract:
  95. - Exact status codes returned (`res.status(201)` / OpenAPI `responses` keys)
  96. - Response construction (`res.json({ data: ... })` / OpenAPI `schema`)
  97. - Error handling paths (what status codes for what conditions)
  98. 2. **READ provider type/model/DTO definitions**: Find the response type referenced by the handler or OpenAPI `$ref` schemas. Extract:
  99. - Exact field names (`transaction_id` not `transactionId`)
  100. - Field types (`string` ID vs `number` ID / OpenAPI `type` + `format`)
  101. - Optional vs required fields (OpenAPI `required` array)
  102. - Nested object structures (OpenAPI `$ref`, `allOf`, `oneOf`)
  103. 3. **READ provider validation schemas**: Find Joi/Zod/class-validator schemas or OpenAPI request body `schema.required`. Extract:
  104. - Required request fields and headers
  105. - Enum/union type allowed values (`"active" | "inactive"` / OpenAPI `enum`)
  106. - Request body constraints
  107. 4. **Cross-reference findings** against consumer expectations:
  108. - Does the consumer expect the same field names the provider sends?
  109. - Does the consumer expect the same status codes the provider returns?
  110. - Does the consumer expect the same nesting the provider produces?
  111. 5. **Document scrutiny evidence** as a block comment in the generated test:
  112. ```typescript
  113. /*
  114. * Provider Scrutiny Evidence:
  115. * - Handler: server/src/routes/userHandlers.ts:45
  116. * - OpenAPI: server/openapi.yaml paths./api/v2/users/{userId}.get (if available)
  117. * - Response type: UserResponseDto (server/src/types/user.ts:12)
  118. * - Status: 201 for creation (line 52), 400 for validation error (line 48)
  119. * - Fields: { id: number, name: string, email: string, role: "user" | "admin" }
  120. * - Required request headers: Authorization (Bearer token)
  121. */
  122. ```
  123. 6. **Graceful degradation** when provider source is not accessible (follows the canonical four-step protocol from `contract-testing.md`):
  124. 1. **OpenAPI/Swagger spec available**: Use the spec as the source of truth for response shapes, status codes, and field names
  125. 2. **Pact Broker available** (when `pact_mcp` is `"mcp"` in `subagentContext.config`): Use SmartBear MCP tools to fetch existing provider states and verified interactions as reference
  126. 3. **Neither available**: Generate from consumer types but use the TODO form of the mandatory comment: `// Provider endpoint: TODO — provider source not accessible, verify manually`. Set `provider_scrutiny: "pending"` in output JSON
  127. 4. **Never silently guess**: Document all assumptions in the scrutiny evidence block
  128. > ⚠️ **Anti-pattern**: Generating response matchers from consumer-side types alone. This produces contracts that reflect what the consumer _wishes_ the provider returns, not what it _actually_ returns. Always read provider source or OpenAPI spec first.
  129. ### 3. Track Fixture Needs
  130. Identify fixtures needed for API tests:
  131. - Authentication fixtures (auth tokens, API keys)
  132. - Data factories (user data, product data, etc.)
  133. - API client configurations
  134. **Do NOT create fixtures yet** - just track what's needed for aggregation step.
  135. ---
  136. ## OUTPUT FORMAT
  137. Write JSON to temp file: `/tmp/tea-automate-api-tests-{{timestamp}}.json`
  138. ```json
  139. {
  140. "success": true,
  141. "subagent": "api-tests",
  142. "tests": [
  143. {
  144. "file": "tests/api/auth.spec.ts",
  145. "content": "[full TypeScript test file content]",
  146. "description": "API tests for authentication endpoints",
  147. "priority_coverage": {
  148. "P0": 3,
  149. "P1": 2,
  150. "P2": 1,
  151. "P3": 0
  152. }
  153. },
  154. {
  155. "file": "tests/api/checkout.spec.ts",
  156. "content": "[full TypeScript test file content]",
  157. "description": "API tests for checkout endpoints",
  158. "priority_coverage": {
  159. "P0": 2,
  160. "P1": 3,
  161. "P2": 1,
  162. "P3": 0
  163. }
  164. }
  165. ],
  166. "fixture_needs": ["authToken", "userDataFactory", "productDataFactory"],
  167. "knowledge_fragments_used": ["api-request", "data-factories", "api-testing-patterns"],
  168. "provider_scrutiny": "completed",
  169. "provider_files_read": ["server/src/routes/authHandlers.ts", "server/src/routes/checkoutHandlers.ts", "server/src/types/auth.ts"],
  170. "test_count": 12,
  171. "summary": "Generated 12 API test cases covering 3 features"
  172. }
  173. ```
  174. **On Error:**
  175. ```json
  176. {
  177. "success": false,
  178. "subagent": "api-tests",
  179. "error": "Error message describing what went wrong",
  180. "partial_output": {
  181. /* any tests generated before error */
  182. }
  183. }
  184. ```
  185. ---
  186. ## EXIT CONDITION
  187. Subagent completes when:
  188. - ✅ All API endpoints have test files generated
  189. - ✅ All tests follow knowledge fragment patterns
  190. - ✅ JSON output written to temp file
  191. - ✅ Fixture needs tracked
  192. **Subagent terminates here.** Parent workflow will read output and proceed to aggregation.
  193. ---
  194. ## 🚨 SUBAGENT SUCCESS METRICS
  195. ### ✅ SUCCESS:
  196. - All API tests generated following patterns
  197. - JSON output valid and complete
  198. - No E2E/component/unit tests included (out of scope)
  199. - Every Pact interaction has `// Provider endpoint:` comment (if CDC enabled)
  200. - Provider source scrutiny completed or gracefully degraded with TODO markers (if CDC enabled)
  201. - Scrutiny evidence documented as block comments in test files (if CDC enabled)
  202. ### ❌ FAILURE:
  203. - Generated tests other than API tests
  204. - Did not follow knowledge fragment patterns
  205. - Invalid or missing JSON output
  206. - Ran tests (not subagent responsibility)
  207. - Pact interactions missing provider endpoint comments (if CDC enabled)
  208. - Response matchers generated from consumer-side types without provider scrutiny (if CDC enabled)