Vous ne pouvez pas sélectionner plus de 25 sujets Les noms de sujets doivent commencer par une lettre ou un nombre, peuvent contenir des tirets ('-') et peuvent comporter jusqu'à 35 caractères.

step-03-gather-evidence.md 3.1KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108
  1. ---
  2. name: 'step-03-gather-evidence'
  3. description: 'Collect evidence for each NFR category'
  4. nextStepFile: '{skill-root}/steps-c/step-04-evaluate-and-score.md'
  5. outputFile: '{test_artifacts}/nfr-assessment.md'
  6. ---
  7. # Step 3: Gather Evidence
  8. ## STEP GOAL
  9. Collect measurable evidence to evaluate each NFR category.
  10. ## MANDATORY EXECUTION RULES
  11. - 📖 Read the entire step file before acting
  12. - ✅ Speak in `{communication_language}`
  13. ---
  14. ## EXECUTION PROTOCOLS:
  15. - 🎯 Follow the MANDATORY SEQUENCE exactly
  16. - 💾 Record outputs before proceeding
  17. - 📖 Load the next step only when instructed
  18. ## CONTEXT BOUNDARIES:
  19. - Available context: config, loaded artifacts, and knowledge fragments
  20. - Focus: this step's goal only
  21. - Limits: do not execute future steps
  22. - Dependencies: prior steps' outputs (if any)
  23. ## MANDATORY SEQUENCE
  24. **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
  25. ## 1. Evidence Sources
  26. Collect evidence for:
  27. - **Performance**: load tests, metrics, response time data
  28. - **Security**: scans, auth tests, vuln reports
  29. - **Reliability**: error rates, burn-in runs, failover tests
  30. - **Maintainability**: test quality, code health signals
  31. - **Other categories**: logs, monitoring, DR drills, deployability checks
  32. ---
  33. ## 2. Browser-Based Evidence Collection (if `tea_browser_automation` is `cli` or `auto`)
  34. > **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip browser-based evidence collection.
  35. For performance and security categories, CLI can gather live evidence:
  36. **Performance evidence (page load, response times):**
  37. 1. `playwright-cli -s=tea-nfr open <target_url>`
  38. 2. `playwright-cli -s=tea-nfr network` → capture response times and payload sizes
  39. 3. `playwright-cli -s=tea-nfr screenshot --filename={test_artifacts}/nfr/perf-<page>.png`
  40. 4. `playwright-cli -s=tea-nfr close`
  41. > **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-nfr close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
  42. Store artifacts under `{test_artifacts}/nfr/`
  43. ---
  44. ## 3. Evidence Gaps
  45. If evidence is missing for a category, mark that category as **CONCERNS**.
  46. ---
  47. ## 4. Save Progress
  48. **Save this step's accumulated work to `{outputFile}`.**
  49. - **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
  50. ```yaml
  51. ---
  52. stepsCompleted: ['step-03-gather-evidence']
  53. lastStep: 'step-03-gather-evidence'
  54. lastSaved: '{date}'
  55. ---
  56. ```
  57. Then write this step's output below the frontmatter.
  58. - **If `{outputFile}` already exists**, update:
  59. - Add `'step-03-gather-evidence'` to `stepsCompleted` array (only if not already present)
  60. - Set `lastStep: 'step-03-gather-evidence'`
  61. - Set `lastSaved: '{date}'`
  62. - Append this step's output to the appropriate section of the document.
  63. Load next step: `{nextStepFile}`
  64. ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
  65. ### ✅ SUCCESS:
  66. - Step completed in full with required outputs
  67. ### ❌ SYSTEM FAILURE:
  68. - Skipped sequence steps or missing outputs
  69. **Master Rule:** Skipping steps is FORBIDDEN.