// End-to-End Quality Strategy
One order, every layer: a quality strategy for an event-driven app
Product risk, critical workflows, test layers from UI to Kafka, CI tiers, and what the suite actually caught — on a public framework of 154 Playwright tests.
Public practice project: a shop app built to be tested (Express, Postgres, Kafka, WebSocket, GraphQL), with a Playwright + TypeScript framework covering every layer. Work projects follow the same approach; their code isn't public.
The product risk
A single order touches five systems: the UI, a REST/GraphQL API, Postgres, a Kafka event, and a WebSocket push back to the user. The expensive failures live between those layers — an order that's saved but never announced, stock that goes negative under concurrent orders, events that arrive out of order, or a real-time update that reaches the wrong user.
UI tests alone can't see most of that, so the strategy puts each risk at the layer where it can be checked directly and cheaply, and keeps only a thin end-to-end layer to prove the layers agree.
Critical workflows
- Login and session handling, including token tampering and expired or missing auth.
- Product search and catalog.
- Cart to order placement, including stock under concurrent orders.
- Order events on Kafka and real-time order updates over WebSocket.
Test layers
| Layer | Tests | What it proves |
|---|---|---|
| UI | 27 | User flows through page objects, axe WCAG 2.1 AA checks, XSS and token-exposure checks; the smoke set also runs on Firefox, WebKit, Pixel 7, and iPhone 14 |
| API (REST + GraphQL) | 84 | CRUD and 400/401/403/404/409 error paths, OpenAPI contract validation, a GraphQL schema snapshot, JWT tampering, SQL injection, mass assignment, rate limiting, and 20 parallel orders racing for 5 units of stock |
| Database | 9 | API calls persist correctly, failed writes roll back with no partial data, stock decrements, passwords are hashed, foreign keys hold |
| Kafka | 4 | Event payload schema, message key equals the order id, events arrive in order, rejected orders publish nothing |
| WebSocket | 11 | Auth, per-user pushes, event order, and no replay of pushes missed while disconnected |
| End to end | 2 | One order traced UI → API → DB → Kafka → WebSocket → UI, with a step per layer |
| Opt-in | — | p95 latency budgets, visual diffs, chaos tests that stop Kafka mid-flow, and a k6 smoke script |
How risk decides what runs when
- A 34-test @smoke tier — the critical set across every layer — runs on every pull request and push to main.
- The full regression suite runs nightly and on demand; cross-browser and mobile runs are limited to the smoke tier, where they earn their cost.
- API and UI suites are sharded, and every test creates its own data and matches events by order id, so parallel runs never collide.
- Performance, visual, and chaos tests are opt-in and single-worker, because they're slow and intentionally disruptive.
What the suite caught
The chaos tests stop the Kafka broker mid-flow. One of them exposed a real defect: when Kafka is down, the order is saved but the API returns a 500 — so the user sees a failure for an order that actually went through, and may place it again.
Rather than deleting or skipping the test, it's kept as an expected failure that documents the bug. The day the API is fixed to handle a broker outage, the suite flags it.
Measurable outcome
- 154 tests across 24 files, covering every layer of the app.
- Nightly full runs take roughly 3 to 15 minutes end to end, and most of that is Docker start-up — the tests themselves take seconds per layer (84 API tests in under 8 seconds across two shards).
- As of October 2026, the last 20 CI runs are green. Tests that pass only on retry are listed in a pull-request comment instead of being hidden by the retry.
- Results publish as merged HTML, JUnit, and Allure reports.
AI in the loop
The repo includes a Playwright MCP setup with planner, generator, and healer agents for drafting and repairing tests, with human review before anything is committed — the same workflow described on the AI-Powered QA page.