Post-Deploy Smoke Tests

Merging a PR is not deploying. Deploying is not working. The gap between "merged" and "working in prod" is where deploy configuration mistakes live — missing env vars, wrong service hostnames, firewall rules that didn't carry over, a database migration that ran on staging but failed silently on production. A full test suite running in CI cannot catch any of these. It never touches the deployed instance.

A smoke suite does.

What a Smoke Suite Is

A smoke suite is a small set of tests — typically 5 to 15 — that run against the real deployed serving surface after each deploy and before any traffic switch. The tests are tagged @smoke so they can run independently of the full regression suite. They use no mocks and no stubs. They hit the actual deployed URL over HTTP, with a real auth token, and verify the critical paths respond correctly.

The design contract is strict:

• Real surface only. No localhost, no test doubles. The tests hit the deployed URL.
• Critical happy paths only. Not edge cases. Not error paths. The minimum set that, if broken, means the service is non-functional.
• Fast. 30 to 60 seconds end to end. A smoke suite that takes 8 minutes is not useful as a deploy gate.
• Blocking. A red smoke cancels the rollout. The new version does not go live.

Designing the Suite

A smoke suite for most services covers five patterns:

1. Health endpoint           GET /health → 200
2. Authentication            GET /api/me → 200 (with real token)
3. Core read path            GET /api/items → 200 (list structure present)
4. Core write path           POST /api/items → 201 + id (probe record)
5. Probe cleanup             DELETE /api/items/{id} → 204, then GET → 404

Pattern 5 — the write-then-delete pair — is covered in depth in the next lesson. For now, note that it closes a critical gap: a health endpoint can return 200 while every write to the database silently fails. The probe write does not.

The @smoke Tag Pattern in Playwright

Playwright (an end-to-end browser/API test runner) drives real requests against a deployed URL, which is exactly what a smoke gate needs. Its --grep flag selects tests by name substring. Naming your smoke tests with @smoke in the test string gives you a free selector:

// Run with: BASE_URL=https://staging.example.com npx playwright test --grep @smoke
import { test, expect } from "@playwright/test"

const BASE_URL = process.env.BASE_URL
if (!BASE_URL) throw new Error("BASE_URL env var required")

test("@smoke health endpoint responds 200", async () => {
  const res = await fetch(`${BASE_URL}/health`)
  expect(res.status).toBe(200)
})

test("@smoke authenticated request is accepted", async () => {
  const res = await fetch(`${BASE_URL}/api/me`, {
    headers: { Authorization: `Bearer ${process.env.SMOKE_AUTH_TOKEN}` },
  })
  expect(res.status).toBe(200)
})

The full regression suite and the smoke suite live in the same file. Untagged tests run in full CI (the merge gate). Tagged @smoke tests run at deploy time (the deploy gate). No duplication.

Wiring Smoke into the Deploy Pipeline

The smoke suite is only useful if it blocks the rollout when it fails. Wire it into the deploy script:

#!/usr/bin/env bash
set -euo pipefail

# 1. Deploy the new version to the target environment
deploy_to_staging

# 2. Run smoke suite — set -e propagates a non-zero exit
BASE_URL="$STAGING_URL" \
SMOKE_AUTH_TOKEN="$STAGING_TOKEN" \
npx playwright test --grep @smoke

# If smoke fails, this line never runs
promote_to_production

set -euo pipefail ensures that a non-zero exit from Playwright aborts the script. The promote step never runs. The rollout is blocked.

Smoke vs Full Suite — The Tradeoff

The table makes the tradeoff concrete: smoke is fast, hits the real surface, and catches deploy-configuration failures. The full suite is thorough, runs in-process, and catches logic regressions. They are not competing — they are complementary. The smoke suite gates the rollout. The full suite gates the merge.

A common mistake is letting the full suite grow into the smoke gate's role: adding mocked smoke tests to CI, running them on every commit, and calling it "covered." This produces a false sense of safety. The mocked tests pass because the mocks always work. The real serving surface is never exercised. A broken deploy config — the failure class smoke tests exist to catch — still reaches users.

Keeping the Suite Honest

Three disciplines keep a smoke suite honest over time:

1. No mocks. Any mock added to a smoke test removes the test from the deployed-surface verification it was designed to provide. Enforce this in code review.
2. Smoke failures get fixed first. A red smoke is treated as a P0 (the highest-priority incident class: drop everything and fix it now). Not "we'll fix it next sprint." The rollout is blocked until smoke is green.
3. Add smoke for every new critical path. When a new endpoint becomes load-bearing for users, add a @smoke test for it. The suite should track the service's critical surface, not stay frozen at the initial 5 tests.

The Inline Rule

A deploy pipeline that can reach production without a passing smoke gate is operationally unsafe. The code quality and the deployed instance are separate concerns, and each deserves its own gate. Smoke is the gate for the deployed instance. Make it non-bypassable.