You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

contract-testing.md 39KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066
  1. # Contract Testing Essentials (Pact)
  2. ## Principle
  3. Contract testing validates API contracts between consumer and provider services without requiring integrated end-to-end tests. Store consumer contracts alongside integration specs, version contracts semantically, and publish on every CI run. Provider verification before merge surfaces breaking changes immediately, while explicit fallback behavior (timeouts, retries, error payloads) captures resilience guarantees in contracts.
  4. > **Pact.js Utils Note**: When `tea_use_pactjs_utils` is enabled, prefer the patterns in the `pactjs-utils-*.md` fragments over the raw Pact.js patterns shown below. The pactjs-utils library eliminates boilerplate for provider states, verifier configuration, and request filters. See `pactjs-utils-overview.md` for the decision tree.
  5. ## Rationale
  6. Traditional integration testing requires running both consumer and provider simultaneously, creating slow, flaky tests with complex setup. Contract testing decouples services: consumers define expectations (pact files), providers verify against those expectations independently. This enables parallel development, catches breaking changes early, and documents API behavior as executable specifications. Pair contract tests with API smoke tests to validate data mapping and UI rendering in tandem.
  7. > **Recommended**: When `tea_use_pactjs_utils` is enabled, use `@seontechnologies/pactjs-utils` utilities instead of the manual patterns below. The library handles JsonMap conversion, verifier configuration, and request filter assembly automatically. See the `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, and `pactjs-utils-request-filter.md` fragments for the simplified approach.
  8. ## Pattern Examples
  9. ### Example 1: Pact Consumer Test (Frontend → Backend API)
  10. **Context**: React application consuming a user management API, defining expected interactions.
  11. **Implementation**:
  12. ```typescript
  13. // tests/contract/user-api.pact.spec.ts
  14. import { PactV3, MatchersV3 } from '@pact-foundation/pact';
  15. import { getUserById, createUser, User } from '@/api/user-service';
  16. const { like, eachLike, string, integer } = MatchersV3;
  17. /**
  18. * Consumer-Driven Contract Test
  19. * - Consumer (React app) defines expected API behavior
  20. * - Generates pact file for provider to verify
  21. * - Runs in isolation (no real backend required)
  22. */
  23. const provider = new PactV3({
  24. consumer: 'user-management-web',
  25. provider: 'user-api-service',
  26. dir: './pacts', // Output directory for pact files
  27. logLevel: 'warn',
  28. });
  29. describe('User API Contract', () => {
  30. describe('GET /users/:id', () => {
  31. it('should return user when user exists', async () => {
  32. // Arrange: Define expected interaction
  33. await provider
  34. .given('user with id 1 exists') // Provider state
  35. .uponReceiving('a request for user 1')
  36. .withRequest({
  37. method: 'GET',
  38. path: '/users/1',
  39. headers: {
  40. Accept: 'application/json',
  41. Authorization: like('Bearer token123'), // Matcher: any string
  42. },
  43. })
  44. .willRespondWith({
  45. status: 200,
  46. headers: {
  47. 'Content-Type': 'application/json',
  48. },
  49. body: like({
  50. id: integer(1),
  51. name: string('John Doe'),
  52. email: string('john@example.com'),
  53. role: string('user'),
  54. createdAt: string('2025-01-15T10:00:00Z'),
  55. }),
  56. })
  57. .executeTest(async (mockServer) => {
  58. // Act: Call consumer code against mock server
  59. const user = await getUserById(1, {
  60. baseURL: mockServer.url,
  61. headers: { Authorization: 'Bearer token123' },
  62. });
  63. // Assert: Validate consumer behavior
  64. expect(user).toEqual(
  65. expect.objectContaining({
  66. id: 1,
  67. name: 'John Doe',
  68. email: 'john@example.com',
  69. role: 'user',
  70. }),
  71. );
  72. });
  73. });
  74. it('should handle 404 when user does not exist', async () => {
  75. await provider
  76. .given('user with id 999 does not exist')
  77. .uponReceiving('a request for non-existent user')
  78. .withRequest({
  79. method: 'GET',
  80. path: '/users/999',
  81. headers: { Accept: 'application/json' },
  82. })
  83. .willRespondWith({
  84. status: 404,
  85. headers: { 'Content-Type': 'application/json' },
  86. body: {
  87. error: 'User not found',
  88. code: 'USER_NOT_FOUND',
  89. },
  90. })
  91. .executeTest(async (mockServer) => {
  92. // Act & Assert: Consumer handles 404 gracefully
  93. await expect(getUserById(999, { baseURL: mockServer.url })).rejects.toThrow('User not found');
  94. });
  95. });
  96. });
  97. describe('POST /users', () => {
  98. it('should create user and return 201', async () => {
  99. const newUser: Omit<User, 'id' | 'createdAt'> = {
  100. name: 'Jane Smith',
  101. email: 'jane@example.com',
  102. role: 'admin',
  103. };
  104. await provider
  105. .given('no users exist')
  106. .uponReceiving('a request to create a user')
  107. .withRequest({
  108. method: 'POST',
  109. path: '/users',
  110. headers: {
  111. 'Content-Type': 'application/json',
  112. Accept: 'application/json',
  113. },
  114. body: newUser,
  115. })
  116. .willRespondWith({
  117. status: 201,
  118. headers: { 'Content-Type': 'application/json' },
  119. body: like({
  120. id: integer(2),
  121. name: string('Jane Smith'),
  122. email: string('jane@example.com'),
  123. role: string('admin'),
  124. createdAt: string('2025-01-15T11:00:00Z'),
  125. }),
  126. })
  127. .executeTest(async (mockServer) => {
  128. const createdUser = await createUser(newUser, {
  129. baseURL: mockServer.url,
  130. });
  131. expect(createdUser).toEqual(
  132. expect.objectContaining({
  133. id: expect.any(Number),
  134. name: 'Jane Smith',
  135. email: 'jane@example.com',
  136. role: 'admin',
  137. }),
  138. );
  139. });
  140. });
  141. });
  142. });
  143. ```
  144. **package.json scripts** (when using pactjs-utils conventions, prefer `test:pact:consumer` naming — see `pact-consumer-framework-setup.md`):
  145. ```json
  146. {
  147. "scripts": {
  148. "test:pact:consumer": "vitest run --config vitest.config.pact.ts",
  149. "publish:pact": ". ./scripts/env-setup.sh && ./scripts/publish-pact.sh"
  150. }
  151. }
  152. ```
  153. **Key Points**:
  154. - **Consumer-driven**: Frontend defines expectations, not backend
  155. - **Matchers (Postel's Law)**: Use `like`, `string`, `integer` matchers in `willRespondWith` (responses) for flexible matching. Do NOT use `like()` on request bodies in `withRequest` — the consumer controls what it sends, so request bodies should use exact values. This follows Postel's Law: be strict in what you send (requests), be lenient in what you accept (responses).
  156. - **Provider states**: given() sets up test preconditions
  157. - **Isolation**: No real backend needed, runs fast
  158. - **Pact generation**: Automatically creates JSON pact files
  159. ---
  160. ### Example 2: Pact Provider Verification (Backend validates contracts)
  161. **Context**: Node.js/Express API verifying pacts published by consumers.
  162. **Implementation**:
  163. ```typescript
  164. // tests/contract/user-api.provider.spec.ts
  165. import { Verifier, VerifierOptions } from '@pact-foundation/pact';
  166. import { server } from '../../src/server'; // Your Express/Fastify app
  167. import { seedDatabase, resetDatabase } from '../support/db-helpers';
  168. /**
  169. * Provider Verification Test
  170. * - Provider (backend API) verifies against published pacts
  171. * - State handlers setup test data for each interaction
  172. * - Runs before merge to catch breaking changes
  173. */
  174. describe('Pact Provider Verification', () => {
  175. let serverInstance;
  176. const PORT = 3001;
  177. beforeAll(async () => {
  178. // Start provider server
  179. serverInstance = server.listen(PORT);
  180. console.log(`Provider server running on port ${PORT}`);
  181. });
  182. afterAll(async () => {
  183. // Cleanup
  184. await serverInstance.close();
  185. });
  186. it('should verify pacts from all consumers', async () => {
  187. const opts: VerifierOptions = {
  188. // Provider details
  189. provider: 'user-api-service',
  190. providerBaseUrl: `http://localhost:${PORT}`,
  191. // Pact Broker configuration
  192. pactBrokerUrl: process.env.PACT_BROKER_BASE_URL,
  193. pactBrokerToken: process.env.PACT_BROKER_TOKEN,
  194. publishVerificationResult: process.env.CI === 'true',
  195. providerVersion: process.env.GITHUB_SHA || 'dev',
  196. // State handlers: Setup provider state for each interaction
  197. stateHandlers: {
  198. 'user with id 1 exists': async () => {
  199. await seedDatabase({
  200. users: [
  201. {
  202. id: 1,
  203. name: 'John Doe',
  204. email: 'john@example.com',
  205. role: 'user',
  206. createdAt: '2025-01-15T10:00:00Z',
  207. },
  208. ],
  209. });
  210. return 'User seeded successfully';
  211. },
  212. 'user with id 999 does not exist': async () => {
  213. // Ensure user doesn't exist
  214. await resetDatabase();
  215. return 'Database reset';
  216. },
  217. 'no users exist': async () => {
  218. await resetDatabase();
  219. return 'Database empty';
  220. },
  221. },
  222. // Request filters: Add auth headers to all requests
  223. requestFilter: (req, res, next) => {
  224. // Mock authentication for verification
  225. req.headers['x-user-id'] = 'test-user';
  226. req.headers['authorization'] = 'Bearer valid-test-token';
  227. next();
  228. },
  229. // Timeout for verification
  230. timeout: 30000,
  231. };
  232. // Run verification
  233. await new Verifier(opts).verifyProvider();
  234. });
  235. });
  236. ```
  237. **CI integration**:
  238. ```yaml
  239. # .github/workflows/contract-test-provider.yml
  240. # NOTE: Canonical naming is contract-test-provider.yml per pactjs-utils conventions
  241. name: Pact Provider Verification
  242. on:
  243. pull_request:
  244. push:
  245. branches: [main]
  246. jobs:
  247. verify-contracts:
  248. runs-on: ubuntu-latest
  249. steps:
  250. - uses: actions/checkout@v4
  251. - name: Setup Node.js
  252. uses: actions/setup-node@v4
  253. with:
  254. node-version-file: '.nvmrc'
  255. - name: Install dependencies
  256. run: npm ci
  257. - name: Start database
  258. run: docker-compose up -d postgres
  259. - name: Run migrations
  260. run: npm run db:migrate
  261. - name: Verify pacts
  262. run: npm run test:pact:provider:remote:contract
  263. env:
  264. PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
  265. PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  266. GITHUB_SHA: ${{ github.sha }}
  267. GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }}
  268. - name: Can I Deploy?
  269. if: github.ref == 'refs/heads/main'
  270. run: npm run can:i:deploy:provider
  271. ```
  272. **Key Points**:
  273. - **State handlers**: Setup provider data for each given() state
  274. - **Request filters**: Add auth/headers for verification requests
  275. - **CI publishing**: Verification results sent to broker
  276. - **can-i-deploy**: Safety check before production deployment
  277. - **Database isolation**: Reset between state handlers
  278. ---
  279. ### Example 3: Contract CI Integration (Consumer & Provider Workflow)
  280. **Context**: Simplified overview of consumer and provider CI coordination. For the complete consumer CI workflow with env blocks, concurrency, and breaking-change detection, see `pact-consumer-framework-setup.md` Example 5.
  281. **Implementation**:
  282. ```yaml
  283. # .github/workflows/contract-test-consumer.yml (Consumer side)
  284. # NOTE: Canonical naming is contract-test-consumer.yml per pactjs-utils conventions
  285. name: Pact Consumer Tests
  286. on:
  287. pull_request:
  288. push:
  289. branches: [main]
  290. jobs:
  291. consumer-tests:
  292. runs-on: ubuntu-latest
  293. steps:
  294. - uses: actions/checkout@v4
  295. - name: Setup Node.js
  296. uses: actions/setup-node@v4
  297. with:
  298. node-version-file: '.nvmrc'
  299. - name: Install dependencies
  300. run: npm ci
  301. - name: Run consumer contract tests
  302. run: npm run test:pact:consumer
  303. - name: Publish pacts to broker
  304. run: npm run publish:pact
  305. - name: Can I deploy consumer? (main only)
  306. if: github.ref == 'refs/heads/main' && env.PACT_BREAKING_CHANGE != 'true'
  307. run: npm run can:i:deploy:consumer
  308. - name: Record consumer deployment (main only)
  309. if: github.ref == 'refs/heads/main'
  310. run: npm run record:consumer:deployment --env=dev
  311. ```
  312. ```yaml
  313. # .github/workflows/contract-test-provider.yml (Provider side)
  314. # NOTE: Canonical naming is contract-test-provider.yml per pactjs-utils conventions
  315. name: Pact Provider Verification
  316. on:
  317. pull_request:
  318. push:
  319. branches: [main]
  320. repository_dispatch:
  321. types: [pact_changed] # Webhook from Pact Broker
  322. jobs:
  323. verify-contracts:
  324. runs-on: ubuntu-latest
  325. steps:
  326. - uses: actions/checkout@v4
  327. - name: Setup Node.js
  328. uses: actions/setup-node@v4
  329. with:
  330. node-version-file: '.nvmrc'
  331. - name: Install dependencies
  332. run: npm ci
  333. - name: Start dependencies
  334. run: docker-compose up -d
  335. - name: Run provider verification
  336. run: npm run test:pact:provider:remote:contract
  337. env:
  338. PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
  339. PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  340. GITHUB_SHA: ${{ github.sha }}
  341. GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }}
  342. - name: Can I deploy provider? (main only)
  343. if: github.ref == 'refs/heads/main' && env.PACT_BREAKING_CHANGE != 'true'
  344. run: npm run can:i:deploy:provider
  345. - name: Record provider deployment (main only)
  346. if: github.ref == 'refs/heads/main'
  347. run: npm run record:provider:deployment --env=dev
  348. ```
  349. **Pact Broker Webhook Configuration**:
  350. ```json
  351. {
  352. "events": [
  353. {
  354. "name": "contract_content_changed"
  355. }
  356. ],
  357. "request": {
  358. "method": "POST",
  359. "url": "https://api.github.com/repos/your-org/user-api/dispatches",
  360. "headers": {
  361. "Authorization": "Bearer ${user.githubToken}",
  362. "Content-Type": "application/json",
  363. "Accept": "application/vnd.github.v3+json"
  364. },
  365. "body": {
  366. "event_type": "pact_changed",
  367. "client_payload": {
  368. "pact_url": "${pactbroker.pactUrl}",
  369. "consumer": "${pactbroker.consumerName}",
  370. "provider": "${pactbroker.providerName}"
  371. }
  372. }
  373. }
  374. }
  375. ```
  376. **Key Points**:
  377. - **Automatic trigger**: Consumer pact changes trigger provider verification via webhook
  378. - **Branch tracking**: Pacts published per branch for feature testing
  379. - **can-i-deploy**: Safety gate before production deployment
  380. - **Record deployment**: Track which version is in each environment
  381. - **Parallel dev**: Consumer and provider teams work independently
  382. ---
  383. ### Example 4: Resilience Coverage (Testing Fallback Behavior)
  384. **Context**: Capture timeout, retry, and error handling behavior explicitly in contracts.
  385. **Implementation**:
  386. ```typescript
  387. // tests/contract/user-api-resilience.pact.spec.ts
  388. import { PactV3, MatchersV3 } from '@pact-foundation/pact';
  389. import { getUserById, ApiError } from '@/api/user-service';
  390. const { like, string } = MatchersV3;
  391. const provider = new PactV3({
  392. consumer: 'user-management-web',
  393. provider: 'user-api-service',
  394. dir: './pacts',
  395. });
  396. describe('User API Resilience Contract', () => {
  397. /**
  398. * Test 500 error handling
  399. * Verifies consumer handles server errors gracefully
  400. */
  401. it('should handle 500 errors with retry logic', async () => {
  402. await provider
  403. .given('server is experiencing errors')
  404. .uponReceiving('a request that returns 500')
  405. .withRequest({
  406. method: 'GET',
  407. path: '/users/1',
  408. headers: { Accept: 'application/json' },
  409. })
  410. .willRespondWith({
  411. status: 500,
  412. headers: { 'Content-Type': 'application/json' },
  413. body: {
  414. error: 'Internal server error',
  415. code: 'INTERNAL_ERROR',
  416. retryable: true,
  417. },
  418. })
  419. .executeTest(async (mockServer) => {
  420. // Consumer should retry on 500
  421. try {
  422. await getUserById(1, {
  423. baseURL: mockServer.url,
  424. retries: 3,
  425. retryDelay: 100,
  426. });
  427. fail('Should have thrown error after retries');
  428. } catch (error) {
  429. expect(error).toBeInstanceOf(ApiError);
  430. expect((error as ApiError).code).toBe('INTERNAL_ERROR');
  431. expect((error as ApiError).retryable).toBe(true);
  432. }
  433. });
  434. });
  435. /**
  436. * Test 429 rate limiting
  437. * Verifies consumer respects rate limits
  438. */
  439. it('should handle 429 rate limit with backoff', async () => {
  440. await provider
  441. .given('rate limit exceeded for user')
  442. .uponReceiving('a request that is rate limited')
  443. .withRequest({
  444. method: 'GET',
  445. path: '/users/1',
  446. })
  447. .willRespondWith({
  448. status: 429,
  449. headers: {
  450. 'Content-Type': 'application/json',
  451. 'Retry-After': '60', // Retry after 60 seconds
  452. },
  453. body: {
  454. error: 'Too many requests',
  455. code: 'RATE_LIMIT_EXCEEDED',
  456. },
  457. })
  458. .executeTest(async (mockServer) => {
  459. try {
  460. await getUserById(1, {
  461. baseURL: mockServer.url,
  462. respectRateLimit: true,
  463. });
  464. fail('Should have thrown rate limit error');
  465. } catch (error) {
  466. expect(error).toBeInstanceOf(ApiError);
  467. expect((error as ApiError).code).toBe('RATE_LIMIT_EXCEEDED');
  468. expect((error as ApiError).retryAfter).toBe(60);
  469. }
  470. });
  471. });
  472. /**
  473. * Test timeout handling
  474. * Verifies consumer has appropriate timeout configuration
  475. */
  476. it('should timeout after 10 seconds', async () => {
  477. await provider
  478. .given('server is slow to respond')
  479. .uponReceiving('a request that times out')
  480. .withRequest({
  481. method: 'GET',
  482. path: '/users/1',
  483. })
  484. .willRespondWith({
  485. status: 200,
  486. headers: { 'Content-Type': 'application/json' },
  487. body: like({ id: 1, name: 'John' }),
  488. })
  489. .withDelay(15000) // Simulate 15 second delay
  490. .executeTest(async (mockServer) => {
  491. try {
  492. await getUserById(1, {
  493. baseURL: mockServer.url,
  494. timeout: 10000, // 10 second timeout
  495. });
  496. fail('Should have timed out');
  497. } catch (error) {
  498. expect(error).toBeInstanceOf(ApiError);
  499. expect((error as ApiError).code).toBe('TIMEOUT');
  500. }
  501. });
  502. });
  503. /**
  504. * Test partial response (optional fields)
  505. * Verifies consumer handles missing optional data
  506. */
  507. it('should handle response with missing optional fields', async () => {
  508. await provider
  509. .given('user exists with minimal data')
  510. .uponReceiving('a request for user with partial data')
  511. .withRequest({
  512. method: 'GET',
  513. path: '/users/1',
  514. })
  515. .willRespondWith({
  516. status: 200,
  517. headers: { 'Content-Type': 'application/json' },
  518. body: {
  519. id: integer(1),
  520. name: string('John Doe'),
  521. email: string('john@example.com'),
  522. // role, createdAt, etc. omitted (optional fields)
  523. },
  524. })
  525. .executeTest(async (mockServer) => {
  526. const user = await getUserById(1, { baseURL: mockServer.url });
  527. // Consumer handles missing optional fields gracefully
  528. expect(user.id).toBe(1);
  529. expect(user.name).toBe('John Doe');
  530. expect(user.role).toBeUndefined(); // Optional field
  531. expect(user.createdAt).toBeUndefined(); // Optional field
  532. });
  533. });
  534. });
  535. ```
  536. **API client with retry logic**:
  537. ```typescript
  538. // src/api/user-service.ts
  539. import axios, { AxiosInstance, AxiosRequestConfig } from 'axios';
  540. export class ApiError extends Error {
  541. constructor(
  542. message: string,
  543. public code: string,
  544. public retryable: boolean = false,
  545. public retryAfter?: number,
  546. ) {
  547. super(message);
  548. }
  549. }
  550. /**
  551. * User API client with retry and error handling
  552. */
  553. export async function getUserById(
  554. id: number,
  555. config?: AxiosRequestConfig & { retries?: number; retryDelay?: number; respectRateLimit?: boolean },
  556. ): Promise<User> {
  557. const { retries = 3, retryDelay = 1000, respectRateLimit = true, ...axiosConfig } = config || {};
  558. let lastError: Error;
  559. for (let attempt = 1; attempt <= retries; attempt++) {
  560. try {
  561. const response = await axios.get(`/users/${id}`, axiosConfig);
  562. return response.data;
  563. } catch (error: any) {
  564. lastError = error;
  565. // Handle rate limiting
  566. if (error.response?.status === 429) {
  567. const retryAfter = parseInt(error.response.headers['retry-after'] || '60');
  568. throw new ApiError('Too many requests', 'RATE_LIMIT_EXCEEDED', false, retryAfter);
  569. }
  570. // Retry on 500 errors
  571. if (error.response?.status === 500 && attempt < retries) {
  572. await new Promise((resolve) => setTimeout(resolve, retryDelay * attempt));
  573. continue;
  574. }
  575. // Handle 404
  576. if (error.response?.status === 404) {
  577. throw new ApiError('User not found', 'USER_NOT_FOUND', false);
  578. }
  579. // Handle timeout
  580. if (error.code === 'ECONNABORTED') {
  581. throw new ApiError('Request timeout', 'TIMEOUT', true);
  582. }
  583. break;
  584. }
  585. }
  586. throw new ApiError('Request failed after retries', 'INTERNAL_ERROR', true);
  587. }
  588. ```
  589. **Key Points**:
  590. - **Resilience contracts**: Timeouts, retries, errors explicitly tested
  591. - **State handlers**: Provider sets up each test scenario
  592. - **Error handling**: Consumer validates graceful degradation
  593. - **Retry logic**: Exponential backoff tested
  594. - **Optional fields**: Consumer handles partial responses
  595. ---
  596. ### Example 5: Pact Broker Housekeeping & Lifecycle Management
  597. **Context**: Automated broker maintenance to prevent contract sprawl and noise.
  598. **Implementation**:
  599. ```typescript
  600. // scripts/pact-broker-housekeeping.ts
  601. /**
  602. * Pact Broker Housekeeping Script
  603. * - Archive superseded contracts
  604. * - Expire unused pacts
  605. * - Tag releases for environment tracking
  606. */
  607. import { execFileSync } from 'node:child_process';
  608. const PACT_BROKER_BASE_URL = process.env.PACT_BROKER_BASE_URL!;
  609. const PACT_BROKER_TOKEN = process.env.PACT_BROKER_TOKEN!;
  610. const PACTICIPANT = 'user-api-service';
  611. /**
  612. * Tag release with environment
  613. */
  614. function tagRelease(version: string, environment: 'staging' | 'production') {
  615. console.log(`🏷️ Tagging ${PACTICIPANT} v${version} as ${environment}`);
  616. execFileSync(
  617. 'pact-broker',
  618. [
  619. 'create-version-tag',
  620. '--pacticipant',
  621. PACTICIPANT,
  622. '--version',
  623. version,
  624. '--tag',
  625. environment,
  626. '--broker-base-url',
  627. PACT_BROKER_BASE_URL,
  628. '--broker-token',
  629. PACT_BROKER_TOKEN,
  630. ],
  631. { stdio: 'inherit' },
  632. );
  633. }
  634. /**
  635. * Record deployment to environment
  636. */
  637. function recordDeployment(version: string, environment: 'staging' | 'production') {
  638. console.log(`📝 Recording deployment of ${PACTICIPANT} v${version} to ${environment}`);
  639. execFileSync(
  640. 'pact-broker',
  641. [
  642. 'record-deployment',
  643. '--pacticipant',
  644. PACTICIPANT,
  645. '--version',
  646. version,
  647. '--environment',
  648. environment,
  649. '--broker-base-url',
  650. PACT_BROKER_BASE_URL,
  651. '--broker-token',
  652. PACT_BROKER_TOKEN,
  653. ],
  654. { stdio: 'inherit' },
  655. );
  656. }
  657. /**
  658. * Clean up old pact versions (retention policy)
  659. * Keep: last 30 days, all production tags, latest from each branch
  660. */
  661. function cleanupOldPacts() {
  662. console.log(`🧹 Cleaning up old pacts for ${PACTICIPANT}`);
  663. execFileSync(
  664. 'pact-broker',
  665. [
  666. 'clean',
  667. '--pacticipant',
  668. PACTICIPANT,
  669. '--broker-base-url',
  670. PACT_BROKER_BASE_URL,
  671. '--broker-token',
  672. PACT_BROKER_TOKEN,
  673. '--keep-latest-for-branch',
  674. '1',
  675. '--keep-min-age',
  676. '30',
  677. ],
  678. { stdio: 'inherit' },
  679. );
  680. }
  681. /**
  682. * Check deployment compatibility
  683. */
  684. function canIDeploy(version: string, toEnvironment: string): boolean {
  685. console.log(`🔍 Checking if ${PACTICIPANT} v${version} can deploy to ${toEnvironment}`);
  686. try {
  687. execFileSync(
  688. 'pact-broker',
  689. [
  690. 'can-i-deploy',
  691. '--pacticipant',
  692. PACTICIPANT,
  693. '--version',
  694. version,
  695. '--to-environment',
  696. toEnvironment,
  697. '--broker-base-url',
  698. PACT_BROKER_BASE_URL,
  699. '--broker-token',
  700. PACT_BROKER_TOKEN,
  701. '--retry-while-unknown',
  702. '10',
  703. '--retry-interval',
  704. '30',
  705. ],
  706. { stdio: 'inherit' },
  707. );
  708. return true;
  709. } catch (error) {
  710. console.error(`❌ Cannot deploy to ${toEnvironment}`);
  711. return false;
  712. }
  713. }
  714. /**
  715. * Main housekeeping workflow
  716. */
  717. async function main() {
  718. const command = process.argv[2];
  719. const version = process.argv[3];
  720. const environment = process.argv[4] as 'staging' | 'production';
  721. switch (command) {
  722. case 'tag-release':
  723. tagRelease(version, environment);
  724. break;
  725. case 'record-deployment':
  726. recordDeployment(version, environment);
  727. break;
  728. case 'can-i-deploy':
  729. const canDeploy = canIDeploy(version, environment);
  730. process.exit(canDeploy ? 0 : 1);
  731. case 'cleanup':
  732. cleanupOldPacts();
  733. break;
  734. default:
  735. console.error('Unknown command. Use: tag-release | record-deployment | can-i-deploy | cleanup');
  736. process.exit(1);
  737. }
  738. }
  739. main();
  740. ```
  741. **package.json scripts**:
  742. ```json
  743. {
  744. "scripts": {
  745. "pact:tag": "ts-node scripts/pact-broker-housekeeping.ts tag-release",
  746. "pact:record": "ts-node scripts/pact-broker-housekeeping.ts record-deployment",
  747. "pact:can-deploy": "ts-node scripts/pact-broker-housekeeping.ts can-i-deploy",
  748. "pact:cleanup": "ts-node scripts/pact-broker-housekeeping.ts cleanup"
  749. }
  750. }
  751. ```
  752. **Deployment workflow integration**:
  753. ```yaml
  754. # .github/workflows/deploy-production.yml
  755. name: Deploy to Production
  756. on:
  757. push:
  758. tags:
  759. - 'v*'
  760. jobs:
  761. verify-contracts:
  762. runs-on: ubuntu-latest
  763. steps:
  764. - uses: actions/checkout@v4
  765. - name: Check pact compatibility
  766. run: npm run pact:can-deploy ${{ github.ref_name }} production
  767. env:
  768. PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
  769. PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  770. deploy:
  771. needs: verify-contracts
  772. runs-on: ubuntu-latest
  773. steps:
  774. - name: Deploy to production
  775. run: ./scripts/deploy.sh production
  776. - name: Record deployment in Pact Broker
  777. run: npm run pact:record ${{ github.ref_name }} production
  778. env:
  779. PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
  780. PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  781. ```
  782. **Scheduled cleanup**:
  783. ```yaml
  784. # .github/workflows/pact-housekeeping.yml
  785. name: Pact Broker Housekeeping
  786. on:
  787. schedule:
  788. - cron: '0 2 * * 0' # Weekly on Sunday at 2 AM
  789. jobs:
  790. cleanup:
  791. runs-on: ubuntu-latest
  792. steps:
  793. - uses: actions/checkout@v4
  794. - name: Cleanup old pacts
  795. run: npm run pact:cleanup
  796. env:
  797. PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
  798. PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  799. ```
  800. **Key Points**:
  801. - **Automated tagging**: Releases tagged with environment
  802. - **Deployment tracking**: Broker knows which version is where
  803. - **Safety gate**: can-i-deploy blocks incompatible deployments
  804. - **Retention policy**: Keep recent, production, and branch-latest pacts
  805. - **Webhook triggers**: Provider verification runs on consumer changes
  806. ---
  807. ## Provider Scrutiny Protocol
  808. When generating consumer contract tests, the agent **MUST** analyze provider source code — or the provider's OpenAPI/Swagger spec — before writing any Pact interaction. Generating contracts from consumer-side assumptions alone leads to mismatches that only surface during provider verification — wrong response shapes, wrong status codes, wrong field names, wrong types, missing required fields, and wrong enum values.
  809. **Source priority**: Provider source code is the most authoritative reference. When an OpenAPI/Swagger spec exists (`openapi.yaml`, `openapi.json`, `swagger.json`), use it as a complementary or alternative source — it documents the provider's contract explicitly and can be faster to parse than tracing through handler code. When both exist, cross-reference them; if they disagree, the source code wins.
  810. ### Provider Endpoint Comment
  811. Every Pact interaction MUST include a provider endpoint comment immediately above the `.given()` call:
  812. ```typescript
  813. // Provider endpoint: server/src/routes/userRouteHandlers.ts -> GET /api/v2/users/:userId
  814. await provider.given('user with id 1 exists').uponReceiving('a request for user 1');
  815. ```
  816. **Format**: `// Provider endpoint: <relative-path-to-handler> -> <METHOD> <route-pattern>`
  817. If the provider source is not accessible, use: `// Provider endpoint: TODO — provider source not accessible, verify manually`
  818. ### Seven-Point Scrutiny Checklist
  819. Before generating each Pact interaction, read the provider route handler and/or OpenAPI spec and verify:
  820. | # | Check | What to Read (source code / OpenAPI spec) | Common Mismatch |
  821. | --- | --------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------- |
  822. | 1 | **Response shape** | Handler's `res.json()` calls / OpenAPI `responses.content.schema` | Nested object vs flat; array wrapper vs direct |
  823. | 2 | **Status codes** | Handler's `res.status()` calls / OpenAPI `responses` keys | 200 vs 201 for creation; 204 vs 200 for delete |
  824. | 3 | **Field names** | Response type/DTO definitions / OpenAPI `schema.properties` | `transaction_id` vs `transactionId`; `fraud_score` vs `score` |
  825. | 4 | **Enum values** | Validation schemas, constants / OpenAPI `schema.enum` | `"active"` vs `"ACTIVE"`; `"pending"` vs `"in_progress"` |
  826. | 5 | **Required fields** | Request validation (Joi, Zod) / OpenAPI `schema.required` | Missing required header; optional field assumed required |
  827. | 6 | **Data types** | TypeScript types, DB models / OpenAPI `schema.type` + `format` | `string` ID vs `number` ID; ISO date vs Unix timestamp |
  828. | 7 | **Nested structures** | Response builder, serializer / OpenAPI `$ref` + `allOf`/`oneOf` | `{ data: { items: [] } }` vs `{ items: [] }` |
  829. ### Scrutiny Evidence Block
  830. Document what was found from provider source and/or OpenAPI spec as a block comment in the test file:
  831. ```typescript
  832. /*
  833. * Provider Scrutiny Evidence:
  834. * - Handler: server/src/routes/userRouteHandlers.ts:45
  835. * - OpenAPI: server/openapi.yaml paths./api/v2/users/{userId}.get (if available)
  836. * - Response type: UserResponseDto (server/src/types/user.ts:12)
  837. * - Status: 200 (line 52), 404 (line 48)
  838. * - Fields: { id: number, name: string, email: string, role: "user" | "admin", createdAt: string }
  839. * - Required request headers: Authorization (Bearer token)
  840. * - Validation: Zod schema at server/src/validation/user.ts:8
  841. */
  842. ```
  843. ### Graceful Degradation
  844. When provider source code is not accessible (different repo, no access, closed source):
  845. 1. **OpenAPI/Swagger spec available**: Use the spec as the source of truth for response shapes, status codes, and field names
  846. 2. **Pact Broker has existing contracts**: Use `pact_mcp` tools to fetch existing provider states and verified interactions as reference
  847. 3. **Neither available**: Generate contracts from consumer-side types but use the TODO form of the mandatory comment: `// Provider endpoint: TODO — provider source not accessible, verify manually` and add a `provider_scrutiny: "pending"` field to the output JSON
  848. 4. **Never silently guess**: If you cannot verify, document what you assumed and why
  849. ---
  850. ## Contract Testing Checklist
  851. Before implementing contract testing, verify:
  852. - [ ] **Pact Broker setup**: Hosted (Pactflow) or self-hosted broker configured
  853. - [ ] **Consumer tests**: Generate pacts in CI, publish to broker on merge
  854. - [ ] **Provider verification**: Runs on PR, verifies all consumer pacts
  855. - [ ] **State handlers**: Provider implements all given() states
  856. - [ ] **can-i-deploy**: Blocks deployment if contracts incompatible
  857. - [ ] **Webhooks configured**: Consumer changes trigger provider verification
  858. - [ ] **Retention policy**: Old pacts archived (keep 30 days, all production tags)
  859. - [ ] **Resilience tested**: Timeouts, retries, error codes in contracts
  860. - [ ] **Provider endpoint comments**: Every Pact interaction has `// Provider endpoint:` comment
  861. - [ ] **Provider scrutiny completed**: Seven-point checklist verified for each interaction
  862. - [ ] **Scrutiny evidence documented**: Block comment with handler, types, status codes, and fields
  863. ## Integration Points
  864. - Used in workflows: `*automate` (integration test generation), `*ci` (contract CI setup)
  865. - Related fragments: `test-levels-framework.md`, `ci-burn-in.md`, `pact-consumer-framework-setup.md` (consumer vitest `fileParallelism: false` + `pool: 'forks'` + `singleFork: true`), `pactjs-utils-consumer-helpers.md` (PactV4 one-interaction-per-`it()` rule), `pactjs-utils-provider-verifier.md` (provider vitest `pool: 'forks'` + `singleFork: true` — same rule as consumer), `pact-broker-webhooks.md` (PactFlow → GitHub webhook auth, PAT rotation, staleness monitoring)
  866. - Tools: Pact.js, Pact Broker (Pactflow or self-hosted), Pact CLI
  867. ---
  868. ## Pact.js Utils Accelerator
  869. When `tea_use_pactjs_utils` is enabled, the following utilities replace manual boilerplate:
  870. | Manual Pattern (raw Pact.js) | Pact.js Utils Equivalent | Benefit |
  871. | -------------------------------------------------------- | --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
  872. | Manual `JsonMap` casting for `.given()` params | `createProviderState({ name, params })` | Type-safe, auto-conversion of Date/null/nested objects |
  873. | Repeated builder callbacks for query/header/body | `setJsonContent({ query, headers, body })` | Reusable callback for `.withRequest(...)` and `.willRespondWith(...)` |
  874. | Inline body lambda `(builder) => builder.jsonBody(body)` | `setJsonBody(body)` | Body-only shorthand for cleaner response builders |
  875. | 30+ lines of `VerifierOptions` assembly | `buildVerifierOptions({ provider, port, includeMainAndDeployed, stateHandlers })` | One-call setup, env-aware, flow auto-detection |
  876. | Manual broker URL + selector logic from env vars | `handlePactBrokerUrlAndSelectors({ ..., options })` | Mutates options in-place with broker URL and selectors |
  877. | DIY Express middleware for auth injection | `createRequestFilter({ tokenGenerator })` | Bearer prefix contract prevents double-prefix bugs |
  878. | Manual CI branch/tag extraction | `getProviderVersionTags()` | CI-aware (GitHub Actions, GitLab CI, etc.) |
  879. | Message verifier config assembly | `buildMessageVerifierOptions({ provider, messageProviders })` | Same one-call pattern for Kafka/async contracts |
  880. | Inline no-op filter `(req, res, next) => next()` | `noOpRequestFilter` | Pre-built pass-through for no-auth providers |
  881. | Hand-written matcher helper duplicating a Zod/TS type | `zodToPactMatchers(ConsumerMovieSchema, example)` | Single source of truth for response shape; consumer-curated scope keeps contracts lean and consumer-driven |
  882. See the `pactjs-utils-*.md` knowledge fragments for complete examples and anti-patterns (`pactjs-utils-zod-to-pact.md` covers the consumer-curated schema pattern).
  883. ### PactV4 Determinism & FFI Safety (Mandatory)
  884. Four rules that together prevent both (a) non-deterministic pact generation failures that cause `Cannot change pact content for already published pact` errors at PactFlow publish, and (b) "request was expected but not received" flakes observed on Linux CI once a consumer+provider pair has more than one `.pacttest.ts` file:
  885. 1. **Consumer Vitest `fileParallelism: false`** in `vitest.config.pact.ts` — prevents parallel workers from racing on the shared pact JSON. See `pact-consumer-framework-setup.md` Example 2.
  886. 2. **Consumer Vitest `pool: 'forks'` + `poolOptions.forks.singleFork: true`** in `vitest.config.pact.ts` — same config as the provider side (`pactjs-utils-provider-verifier.md` Example 7). Best current understanding: the `@pact-foundation/pact` napi-rs binding is not robust across Vitest worker threads sharing a process; serialization alone (via `fileParallelism: false`) is insufficient on the default threads pool in Vitest v1. Forks + `singleFork: true` runs every pact file in one subprocess with a coherent FFI handle and eliminated a reproducible Linux-CI flake on two repos (`pactjs-utils`, `seon-mcp-server`). Single-file consumer suites have not been observed to flake; this rule is still recommended as a future-proof. See `pact-consumer-framework-setup.md` Example 2.
  887. 3. **One `addInteraction()` per `it()` block** — see `pactjs-utils-consumer-helpers.md` Example 6.
  888. 4. **`publish-pact.sh` jq normalization** sorts interactions before publish — ensures byte-stable payload to PactFlow regardless of generator ordering quirks. See `pact-consumer-framework-setup.md` Example 4.
  889. Provider suites require the same `pool: 'forks'` + `singleFork: true` combination — see `pactjs-utils-provider-verifier.md` Example 7.
  890. ### Webhook Auth & Staleness
  891. When `can-i-deploy` in a consumer repo times out with `There is no verified pact between <consumer> and the version of <provider> currently in <env>` — check the provider's PactFlow webhook. Silent failures from an expired/revoked GitHub PAT are the most common non-code cause of this symptom. See `pact-broker-webhooks.md` for the dedicated-machine-user pattern, classic-PAT-with-`repo`-scope rationale, rotation runbook, and staleness monitoring options.
  892. _Source: Pact consumer/provider sample repos, Murat contract testing blog, Pact official documentation, @seontechnologies/pactjs-utils library_