bmad-template/.workbuddy/skills/bmad-tea/resources/knowledge/selective-testing.md

# Selective and Targeted Test Execution

## Principle

Run only the tests you need, when you need them. Use tags/grep to slice suites by risk priority (not directory structure), filter by spec patterns or git diff to focus on impacted areas, and combine priority metadata (P0-P3) with change detection to optimize pre-commit vs. CI execution. Document the selection strategy clearly so teams understand when full regression is mandatory.

## Rationale

Running the entire test suite on every commit wastes time and resources. Smart test selection provides fast feedback (smoke tests in minutes, full regression in hours) while maintaining confidence. The "32+ ways of selective testing" philosophy balances speed with coverage: quick loops for developers, comprehensive validation before deployment. Poorly documented selection leads to confusion about when tests run and why.

## Pattern Examples

### Example 1: Tag-Based Execution with Priority Levels

**Context**: Organize tests by risk priority and execution stage using grep/tag patterns.

**Implementation**:

```typescript
// tests/e2e/checkout.spec.ts
import { test, expect } from '@playwright/test';

/**
 * Tag-based test organization
 * - @smoke: Critical path tests (run on every commit, < 5 min)
 * - @regression: Full test suite (run pre-merge, < 30 min)
 * - @p0: Critical business functions (payment, auth, data integrity)
 * - @p1: Core features (primary user journeys)
 * - @p2: Secondary features (supporting functionality)
 * - @p3: Nice-to-have (cosmetic, non-critical)
 */

test.describe('Checkout Flow', () => {
  // P0 + Smoke: Must run on every commit
  test('@smoke @p0 should complete purchase with valid payment', async ({ page }) => {
    await page.goto('/checkout');
    await page.getByTestId('card-number').fill('4242424242424242');
    await page.getByTestId('submit-payment').click();

    await expect(page.getByTestId('order-confirmation')).toBeVisible();
  });

  // P0 but not smoke: Run pre-merge
  test('@regression @p0 should handle payment decline gracefully', async ({ page }) => {
    await page.goto('/checkout');
    await page.getByTestId('card-number').fill('4000000000000002'); // Decline card
    await page.getByTestId('submit-payment').click();

    await expect(page.getByTestId('payment-error')).toBeVisible();
    await expect(page.getByTestId('payment-error')).toContainText('declined');
  });

  // P1 + Smoke: Important but not critical
  test('@smoke @p1 should apply discount code', async ({ page }) => {
    await page.goto('/checkout');
    await page.getByTestId('promo-code').fill('SAVE10');
    await page.getByTestId('apply-promo').click();

    await expect(page.getByTestId('discount-applied')).toBeVisible();
  });

  // P2: Run in full regression only
  test('@regression @p2 should remember saved payment methods', async ({ page }) => {
    await page.goto('/checkout');
    await expect(page.getByTestId('saved-cards')).toBeVisible();
  });

  // P3: Low priority, run nightly or weekly
  test('@nightly @p3 should display checkout page analytics', async ({ page }) => {
    await page.goto('/checkout');
    const analyticsEvents = await page.evaluate(() => (window as any).__ANALYTICS__);
    expect(analyticsEvents).toBeDefined();
  });
});
```

**package.json scripts**:

```json
{
  "scripts": {
    "test": "playwright test",
    "test:smoke": "playwright test --grep '@smoke'",
    "test:p0": "playwright test --grep '@p0'",
    "test:p0-p1": "playwright test --grep '@p0|@p1'",
    "test:regression": "playwright test --grep '@regression'",
    "test:nightly": "playwright test --grep '@nightly'",
    "test:not-slow": "playwright test --grep-invert '@slow'",
    "test:critical-smoke": "playwright test --grep '@smoke.*@p0'"
  }
}
```

**Cypress equivalent**:

```javascript
// cypress/e2e/checkout.cy.ts
describe('Checkout Flow', { tags: ['@checkout'] }, () => {
  it('should complete purchase', { tags: ['@smoke', '@p0'] }, () => {
    cy.visit('/checkout');
    cy.get('[data-cy="card-number"]').type('4242424242424242');
    cy.get('[data-cy="submit-payment"]').click();
    cy.get('[data-cy="order-confirmation"]').should('be.visible');
  });

  it('should handle decline', { tags: ['@regression', '@p0'] }, () => {
    cy.visit('/checkout');
    cy.get('[data-cy="card-number"]').type('4000000000000002');
    cy.get('[data-cy="submit-payment"]').click();
    cy.get('[data-cy="payment-error"]').should('be.visible');
  });
});

// cypress.config.ts
export default defineConfig({
  e2e: {
    env: {
      grepTags: process.env.GREP_TAGS || '',
      grepFilterSpecs: true,
    },
    setupNodeEvents(on, config) {
      require('@cypress/grep/src/plugin')(config);
      return config;
    },
  },
});
```

**Usage**:

```bash
# Playwright
npm run test:smoke                    # Run all @smoke tests
npm run test:p0                       # Run all P0 tests
npm run test -- --grep "@smoke.*@p0"  # Run tests with BOTH tags

# Cypress (with @cypress/grep plugin)
npx cypress run --env grepTags="@smoke"
npx cypress run --env grepTags="@p0+@smoke"  # AND logic
npx cypress run --env grepTags="@p0 @p1"     # OR logic
```

**Key Points**:

- **Multiple tags per test**: Combine priority (@p0) with stage (@smoke)
- **AND/OR logic**: Grep supports complex filtering
- **Clear naming**: Tags document test importance
- **Fast feedback**: @smoke runs < 5 min, full suite < 30 min
- **CI integration**: Different jobs run different tag combinations

---

### Example 2: Spec Filter Pattern (File-Based Selection)

**Context**: Run tests by file path pattern or directory for targeted execution.

**Implementation**:

```bash
#!/bin/bash
# scripts/selective-spec-runner.sh
# Run tests based on spec file patterns

set -e

PATTERN=${1:-"**/*.spec.ts"}
TEST_ENV=${TEST_ENV:-local}

echo "🎯 Selective Spec Runner"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "Pattern: $PATTERN"
echo "Environment: $TEST_ENV"
echo ""

# Pattern examples and their use cases
case "$PATTERN" in
  "**/checkout*")
    echo "📦 Running checkout-related tests"
    npx playwright test --grep-files="**/checkout*"
    ;;
  "**/auth*"|"**/login*"|"**/signup*")
    echo "🔐 Running authentication tests"
    npx playwright test --grep-files="**/auth*|**/login*|**/signup*"
    ;;
  "tests/e2e/**")
    echo "🌐 Running all E2E tests"
    npx playwright test tests/e2e/
    ;;
  "tests/integration/**")
    echo "🔌 Running all integration tests"
    npx playwright test tests/integration/
    ;;
  "tests/component/**")
    echo "🧩 Running all component tests"
    npx playwright test tests/component/
    ;;
  *)
    echo "🔍 Running tests matching pattern: $PATTERN"
    npx playwright test "$PATTERN"
    ;;
esac
```

**Playwright config for file filtering**:

```typescript
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  // ... other config

  // Project-based organization
  projects: [
    {
      name: 'smoke',
      testMatch: /.*smoke.*\.spec\.ts/,
      retries: 0,
    },
    {
      name: 'e2e',
      testMatch: /tests\/e2e\/.*\.spec\.ts/,
      retries: 2,
    },
    {
      name: 'integration',
      testMatch: /tests\/integration\/.*\.spec\.ts/,
      retries: 1,
    },
    {
      name: 'component',
      testMatch: /tests\/component\/.*\.spec\.ts/,
      use: { ...devices['Desktop Chrome'] },
    },
  ],
});
```

**Advanced pattern matching**:

```typescript
// scripts/run-by-component.ts
/**
 * Run tests related to specific component(s)
 * Usage: npm run test:component UserProfile,Settings
 */

import { execSync } from 'child_process';

const components = process.argv[2]?.split(',') || [];

if (components.length === 0) {
  console.error('❌ No components specified');
  console.log('Usage: npm run test:component UserProfile,Settings');
  process.exit(1);
}

// Convert component names to glob patterns
const patterns = components.map((comp) => `**/*${comp}*.spec.ts`).join(' ');

console.log(`🧩 Running tests for components: ${components.join(', ')}`);
console.log(`Patterns: ${patterns}`);

try {
  execSync(`npx playwright test ${patterns}`, {
    stdio: 'inherit',
    env: { ...process.env, CI: 'false' },
  });
} catch (error) {
  process.exit(1);
}
```

**package.json scripts**:

```json
{
  "scripts": {
    "test:checkout": "playwright test **/checkout*.spec.ts",
    "test:auth": "playwright test **/auth*.spec.ts **/login*.spec.ts",
    "test:e2e": "playwright test tests/e2e/",
    "test:integration": "playwright test tests/integration/",
    "test:component": "ts-node scripts/run-by-component.ts",
    "test:project": "playwright test --project",
    "test:smoke-project": "playwright test --project smoke"
  }
}
```

**Key Points**:

- **Glob patterns**: Wildcards match file paths flexibly
- **Project isolation**: Separate projects have different configs
- **Component targeting**: Run tests for specific features
- **Directory-based**: Organize tests by type (e2e, integration, component)
- **CI optimization**: Run subsets in parallel CI jobs

---

### Example 3: Diff-Based Test Selection (Changed Files Only)

**Context**: Run only tests affected by code changes for maximum speed.

**Implementation**:

```bash
#!/bin/bash
# scripts/test-changed-files.sh
# Intelligent test selection based on git diff

set -e

BASE_BRANCH=${BASE_BRANCH:-main}
TEST_ENV=${TEST_ENV:-local}

echo "🔍 Changed File Test Selector"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "Base branch: $BASE_BRANCH"
echo "Environment: $TEST_ENV"
echo ""

# Get changed files
CHANGED_FILES=$(git diff --name-only $BASE_BRANCH...HEAD)

if [ -z "$CHANGED_FILES" ]; then
  echo "✅ No files changed. Skipping tests."
  exit 0
fi

echo "Changed files:"
echo "$CHANGED_FILES" | sed 's/^/  - /'
echo ""

# Arrays to collect test specs
DIRECT_TEST_FILES=()
RELATED_TEST_FILES=()
RUN_ALL_TESTS=false

# Process each changed file
while IFS= read -r file; do
  case "$file" in
    # Changed test files: run them directly
    *.spec.ts|*.spec.js|*.test.ts|*.test.js|*.cy.ts|*.cy.js)
      DIRECT_TEST_FILES+=("$file")
      ;;

    # Critical config changes: run ALL tests
    package.json|package-lock.json|playwright.config.ts|cypress.config.ts|tsconfig.json|.github/workflows/*)
      echo "⚠️  Critical file changed: $file"
      RUN_ALL_TESTS=true
      break
      ;;

    # Component changes: find related tests
    src/components/*.tsx|src/components/*.jsx)
      COMPONENT_NAME=$(basename "$file" | sed 's/\.[^.]*$//')
      echo "🧩 Component changed: $COMPONENT_NAME"

      # Find tests matching component name
      FOUND_TESTS=$(find tests -name "*${COMPONENT_NAME}*.spec.ts" -o -name "*${COMPONENT_NAME}*.cy.ts" 2>/dev/null || true)
      if [ -n "$FOUND_TESTS" ]; then
        while IFS= read -r test_file; do
          RELATED_TEST_FILES+=("$test_file")
        done <<< "$FOUND_TESTS"
      fi
      ;;

    # Utility/lib changes: run integration + unit tests
    src/utils/*|src/lib/*|src/helpers/*)
      echo "⚙️  Utility file changed: $file"
      RELATED_TEST_FILES+=($(find tests/unit tests/integration -name "*.spec.ts" 2>/dev/null || true))
      ;;

    # API changes: run integration + e2e tests
    src/api/*|src/services/*|src/controllers/*)
      echo "🔌 API file changed: $file"
      RELATED_TEST_FILES+=($(find tests/integration tests/e2e -name "*.spec.ts" 2>/dev/null || true))
      ;;

    # Type changes: run all TypeScript tests
    *.d.ts|src/types/*)
      echo "📝 Type definition changed: $file"
      RUN_ALL_TESTS=true
      break
      ;;

    # Documentation only: skip tests
    *.md|docs/*|README*)
      echo "📄 Documentation changed: $file (no tests needed)"
      ;;

    *)
      echo "❓ Unclassified change: $file (running smoke tests)"
      RELATED_TEST_FILES+=($(find tests -name "*smoke*.spec.ts" 2>/dev/null || true))
      ;;
  esac
done <<< "$CHANGED_FILES"

# Execute tests based on analysis
if [ "$RUN_ALL_TESTS" = true ]; then
  echo ""
  echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  echo "🚨 Running FULL test suite (critical changes detected)"
  echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
  npm run test
  exit $?
fi

# Combine and deduplicate test files
ALL_TEST_FILES=(${DIRECT_TEST_FILES[@]} ${RELATED_TEST_FILES[@]})
UNIQUE_TEST_FILES=($(echo "${ALL_TEST_FILES[@]}" | tr ' ' '\n' | sort -u))

if [ ${#UNIQUE_TEST_FILES[@]} -eq 0 ]; then
  echo ""
  echo "✅ No tests found for changed files. Running smoke tests."
  npm run test:smoke
  exit $?
fi

echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🎯 Running ${#UNIQUE_TEST_FILES[@]} test file(s)"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

for test_file in "${UNIQUE_TEST_FILES[@]}"; do
  echo "  - $test_file"
done

echo ""
npm run test -- "${UNIQUE_TEST_FILES[@]}"
```

**GitHub Actions integration**:

```yaml
# .github/workflows/test-changed.yml
name: Test Changed Files
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  detect-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # Full history for accurate diff

      - name: Get changed files
        id: changed-files
        uses: tj-actions/changed-files@v40
        with:
          files: |
            src/**
            tests/**
            *.config.ts
          files_ignore: |
            **/*.md
            docs/**

      - name: Run tests for changed files
        if: steps.changed-files.outputs.any_changed == 'true'
        run: |
          echo "Changed files: ${{ steps.changed-files.outputs.all_changed_files }}"
          bash scripts/test-changed-files.sh
        env:
          BASE_BRANCH: ${{ github.base_ref }}
          TEST_ENV: staging
```

**Key Points**:

- **Intelligent mapping**: Code changes → related tests
- **Critical file detection**: Config changes = full suite
- **Component mapping**: UI changes → component + E2E tests
- **Fast feedback**: Run only what's needed (< 2 min typical)
- **Safety net**: Unrecognized changes run smoke tests

---

### Example 4: Promotion Rules (Pre-Commit → CI → Staging → Production)

**Context**: Progressive test execution strategy across deployment stages.

**Implementation**:

```typescript
// scripts/test-promotion-strategy.ts
/**
 * Test Promotion Strategy
 * Defines which tests run at each stage of the development lifecycle
 */

export type TestStage = 'pre-commit' | 'ci-pr' | 'ci-merge' | 'staging' | 'production';

export type TestPromotion = {
  stage: TestStage;
  description: string;
  testCommand: string;
  timebudget: string; // minutes
  required: boolean;
  failureAction: 'block' | 'warn' | 'alert';
};

export const TEST_PROMOTION_RULES: Record<TestStage, TestPromotion> = {
  'pre-commit': {
    stage: 'pre-commit',
    description: 'Local developer checks before git commit',
    testCommand: 'npm run test:smoke',
    timebudget: '2',
    required: true,
    failureAction: 'block',
  },
  'ci-pr': {
    stage: 'ci-pr',
    description: 'CI checks on pull request creation/update',
    testCommand: 'npm run test:changed && npm run test:p0-p1',
    timebudget: '10',
    required: true,
    failureAction: 'block',
  },
  'ci-merge': {
    stage: 'ci-merge',
    description: 'Full regression before merge to main',
    testCommand: 'npm run test:regression',
    timebudget: '30',
    required: true,
    failureAction: 'block',
  },
  staging: {
    stage: 'staging',
    description: 'Post-deployment validation in staging environment',
    testCommand: 'npm run test:e2e -- --grep "@smoke"',
    timebudget: '15',
    required: true,
    failureAction: 'block',
  },
  production: {
    stage: 'production',
    description: 'Production smoke tests post-deployment',
    testCommand: 'npm run test:e2e:prod -- --grep "@smoke.*@p0"',
    timebudget: '5',
    required: false,
    failureAction: 'alert',
  },
};

/**
 * Get tests to run for a specific stage
 */
export function getTestsForStage(stage: TestStage): TestPromotion {
  return TEST_PROMOTION_RULES[stage];
}

/**
 * Validate if tests can be promoted to next stage
 */
export function canPromote(currentStage: TestStage, testsPassed: boolean): boolean {
  const promotion = TEST_PROMOTION_RULES[currentStage];

  if (!promotion.required) {
    return true; // Non-required tests don't block promotion
  }

  return testsPassed;
}
```

**Husky pre-commit hook**:

```bash
#!/bin/bash
# .husky/pre-commit
# Run smoke tests before allowing commit

echo "🔍 Running pre-commit tests..."

npm run test:smoke

if [ $? -ne 0 ]; then
  echo ""
  echo "❌ Pre-commit tests failed!"
  echo "Please fix failures before committing."
  echo ""
  echo "To skip (NOT recommended): git commit --no-verify"
  exit 1
fi

echo "✅ Pre-commit tests passed"
```

**GitHub Actions workflow**:

```yaml
# .github/workflows/test-promotion.yml
name: Test Promotion Strategy
on:
  pull_request:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  # Stage 1: PR tests (changed + P0-P1)
  pr-tests:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - name: Run PR-level tests
        run: |
          npm run test:changed
          npm run test:p0-p1

  # Stage 2: Full regression (pre-merge)
  regression-tests:
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
      - name: Run full regression
        run: npm run test:regression

  # Stage 3: Staging validation (post-deploy)
  staging-smoke:
    if: github.event_name == 'workflow_dispatch'
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - name: Run staging smoke tests
        run: npm run test:e2e -- --grep "@smoke"
        env:
          TEST_ENV: staging

  # Stage 4: Production smoke (post-deploy, non-blocking)
  production-smoke:
    if: github.event_name == 'workflow_dispatch'
    runs-on: ubuntu-latest
    timeout-minutes: 5
    continue-on-error: true # Don't fail deployment if smoke tests fail
    steps:
      - uses: actions/checkout@v4
      - name: Run production smoke tests
        run: npm run test:e2e:prod -- --grep "@smoke.*@p0"
        env:
          TEST_ENV: production

      - name: Alert on failure
        if: failure()
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ job.status }}
          text: '🚨 Production smoke tests failed!'
          webhook_url: ${{ secrets.SLACK_WEBHOOK }}
```

**Selection strategy documentation**:

````markdown
# Test Selection Strategy

## Test Promotion Stages

| Stage      | Tests Run           | Time Budget | Blocks Deploy | Failure Action |
| ---------- | ------------------- | ----------- | ------------- | -------------- |
| Pre-Commit | Smoke (@smoke)      | 2 min       | ✅ Yes        | Block commit   |
| CI PR      | Changed + P0-P1     | 10 min      | ✅ Yes        | Block merge    |
| CI Merge   | Full regression     | 30 min      | ✅ Yes        | Block deploy   |
| Staging    | E2E smoke           | 15 min      | ✅ Yes        | Rollback       |
| Production | Critical smoke only | 5 min       | ❌ No         | Alert team     |

## When Full Regression Runs

Full regression suite (`npm run test:regression`) runs in these scenarios:

- ✅ Before merging to `main` (CI Merge stage)
- ✅ Nightly builds (scheduled workflow)
- ✅ Manual trigger (workflow_dispatch)
- ✅ Release candidate testing

Full regression does NOT run on:

- ❌ Every PR commit (too slow)
- ❌ Pre-commit hooks (too slow)
- ❌ Production deployments (deploy-blocking)

## Override Scenarios

Skip tests (emergency only):

```bash
git commit --no-verify  # Skip pre-commit hook
gh pr merge --admin     # Force merge (requires admin)
```
````

```

**Key Points**:
- **Progressive validation**: More tests at each stage
- **Time budgets**: Clear expectations per stage
- **Blocking vs. alerting**: Production tests don't block deploy
- **Documentation**: Team knows when full regression runs
- **Emergency overrides**: Documented but discouraged

---

## Test Selection Strategy Checklist

Before implementing selective testing, verify:

- [ ] **Tag strategy defined**: @smoke, @p0-p3, @regression documented
- [ ] **Time budgets set**: Each stage has clear timeout (smoke < 5 min, full < 30 min)
- [ ] **Changed file mapping**: Code changes → test selection logic implemented
- [ ] **Promotion rules documented**: README explains when full regression runs
- [ ] **CI integration**: GitHub Actions uses selective strategy
- [ ] **Local parity**: Developers can run same selections locally
- [ ] **Emergency overrides**: Skip mechanisms documented (--no-verify, admin merge)
- [ ] **Metrics tracked**: Monitor test execution time and selection accuracy

## Integration Points

- Used in workflows: `*ci` (CI/CD setup), `*automate` (test generation with tags)
- Related fragments: `ci-burn-in.md`, `test-priorities-matrix.md`, `test-quality.md`
- Selection tools: Playwright --grep, Cypress @cypress/grep, git diff

_Source: 32+ selective testing strategies blog, Murat testing philosophy, enterprise CI optimization_
```