Nelze vybrat více než 25 témat Téma musí začínat písmenem nebo číslem, může obsahovat pomlčky („-“) a může být dlouhé až 35 znaků.

step-03-configure-quality-gates.md 5.4KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145
  1. ---
  2. name: 'step-03-configure-quality-gates'
  3. description: 'Configure burn-in, quality gates, and notifications'
  4. nextStepFile: '{skill-root}/steps-c/step-04-validate-and-summary.md'
  5. knowledgeIndex: './resources/tea-index.csv'
  6. outputFile: '{test_artifacts}/ci-pipeline-progress.md'
  7. ---
  8. # Step 3: Quality Gates & Notifications
  9. ## STEP GOAL
  10. Configure burn-in loops, quality thresholds, and notification hooks.
  11. ## MANDATORY EXECUTION RULES
  12. - 📖 Read the entire step file before acting
  13. - ✅ Speak in `{communication_language}`
  14. ---
  15. ## EXECUTION PROTOCOLS:
  16. - 🎯 Follow the MANDATORY SEQUENCE exactly
  17. - 💾 Record outputs before proceeding
  18. - 📖 Load the next step only when instructed
  19. ## CONTEXT BOUNDARIES:
  20. - Available context: config, loaded artifacts, and knowledge fragments
  21. - Focus: this step's goal only
  22. - Limits: do not execute future steps
  23. - Dependencies: prior steps' outputs (if any)
  24. ## MANDATORY SEQUENCE
  25. **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
  26. ## 1. Burn-In Configuration
  27. Use `{knowledgeIndex}` to load `ci-burn-in.md` guidance:
  28. - Run N-iteration burn-in for flaky detection
  29. - Gate promotion based on burn-in stability
  30. **Stack-conditional burn-in:**
  31. - **Frontend or Fullstack** (`test_stack_type` is `frontend` or `fullstack`): Enable burn-in by default. Burn-in targets UI flakiness (race conditions, selector instability, timing issues).
  32. - **Backend only** (`test_stack_type` is `backend`): Skip burn-in by default. Backend tests (unit, integration, API) are deterministic and rarely exhibit UI-related flakiness. If the user explicitly requests burn-in for backend, honor that override.
  33. **Security: Script injection prevention for reusable burn-in workflows:**
  34. When burn-in is extracted into a reusable workflow (`on: workflow_call`), all `${{ inputs.* }}` values MUST be passed through `env:` intermediaries and referenced as quoted `"$ENV_VAR"`. Never interpolate them directly.
  35. **Inputs must be DATA, not COMMANDS.** Do not accept command-shaped inputs (e.g., `inputs.install-command`, `inputs.test-command`) that get executed as shell code — even through `env:`, running `$CMD` is still command injection. Use fixed commands (e.g., `npm ci`, `npx playwright test`) and pass inputs only as data arguments.
  36. ```yaml
  37. # ✅ SAFE — fixed commands with data-only inputs
  38. - name: Install dependencies
  39. run: npm ci
  40. - name: Run burn-in loop
  41. env:
  42. TEST_GREP: ${{ inputs.test-grep }}
  43. BURN_IN_COUNT: ${{ inputs.burn-in-count }}
  44. BASE_REF: ${{ inputs.base-ref }}
  45. run: |
  46. # Security: inputs passed through env: to prevent script injection
  47. for i in $(seq 1 "$BURN_IN_COUNT"); do
  48. echo "Burn-in iteration $i/$BURN_IN_COUNT"
  49. npx playwright test --grep "$TEST_GREP" || exit 1
  50. done
  51. ```
  52. ---
  53. ## 2. Quality Gates
  54. Define:
  55. - Minimum pass rates (P0 = 100%, P1 ≥ 95%)
  56. - Fail CI on critical test failures
  57. - Optional: require traceability or nfr-assess output before release
  58. **Contract testing gate** (if `tea_use_pactjs_utils` is enabled):
  59. Use `{knowledgeIndex}` to load:
  60. - `pact-consumer-framework-setup.md` — determinism gate (`check-pact-determinism.sh`), `jq -S` publish normalization, 1:1 local/CI parity
  61. - `pactjs-utils-consumer-helpers.md` — one-interaction-per-`it()` determinism rule
  62. - `pactjs-utils-provider-verifier.md` — `buildVerifierOptions`, broker config, breaking change patterns, vitest `pool: 'forks'` + `singleFork` (same rule applies to consumer AND provider)
  63. - `pactjs-utils-request-filter.md` — `createRequestFilter` auth injection patterns for CI pipeline auth setup
  64. - `pact-broker-webhooks.md` — webhook auth pattern, PAT rotation runbook, staleness monitoring (webhook failures silently break `can-i-deploy`)
  65. - **Determinism gate must pass** (consumer side): `npm run test:pact:consumer` runs the suite N times and fails on byte-different pact JSON before any publish is attempted. This is a non-negotiable pre-publish gate.
  66. - **can-i-deploy must pass** before any deployment to staging or production
  67. - Block the deployment pipeline if contract verification fails
  68. - Treat consumer pact publishing failures as CI failures (contracts must stay up-to-date)
  69. - Provider verification must pass for all consumer pacts before merge
  70. - **Staleness alert**: scheduled job asserts recent verifications exist — a missing signal indicates a silently-broken webhook (usually an expired GitHub PAT on the PactFlow secret; see `pact-broker-webhooks.md` rotation runbook).
  71. ---
  72. ## 3. Notifications
  73. Configure:
  74. - Failure notifications (Slack/email)
  75. - Artifact links
  76. ---
  77. ### 4. Save Progress
  78. **Save this step's accumulated work to `{outputFile}`.**
  79. - **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
  80. ```yaml
  81. ---
  82. stepsCompleted: ['step-03-configure-quality-gates']
  83. lastStep: 'step-03-configure-quality-gates'
  84. lastSaved: '{date}'
  85. ---
  86. ```
  87. Then write this step's output below the frontmatter.
  88. - **If `{outputFile}` already exists**, update:
  89. - Add `'step-03-configure-quality-gates'` to `stepsCompleted` array (only if not already present)
  90. - Set `lastStep: 'step-03-configure-quality-gates'`
  91. - Set `lastSaved: '{date}'`
  92. - Append this step's output to the appropriate section of the document.
  93. Load next step: `{nextStepFile}`
  94. ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
  95. ### ✅ SUCCESS:
  96. - Step completed in full with required outputs
  97. ### ❌ SYSTEM FAILURE:
  98. - Skipped sequence steps or missing outputs
  99. **Master Rule:** Skipping steps is FORBIDDEN.