Metrics & Impact
The signals I use to make quality decisions, each labeled with where the figure came from and how it feeds a release decision.
// metric ledger
Quality metrics
Critical journey coverage
Measured at work90%+
LiveSwitch · Web, Android, iOS
90%+ automation coverage across Web, Android, and iOS at LiveSwitch; 95%+ test coverage across releases at Mastercard. In public: a 34-test smoke tier covering every critical layer runs on every pull request.
Why it matters
How many of the user journeys that matter most are protected by tests — not raw test count.
Regression duration
Measured at work~40%
LiveSwitch · faster regression runs
~40% lower regression execution time at LiveSwitch, before vs. after the Selenium-to-Playwright migration and AI-assisted workflow.
Why it matters
How fast the team gets a trustworthy answer before a release.
Defect resolution speed
Measured at work~20%
Mastercard · faster resolution
~20% faster defect resolution at Mastercard after introducing structured QA-Dev syncs and better triage.
Why it matters
Whether QA and Dev are closing the loop on what testing finds.
Escaped defects
Measured at work<5%
Triomics · LiveSwitch
Under 5% of defects escaped to production at Triomics and LiveSwitch. Each one that did was traced back to the test that should have caught it, and that test was added to regression so the same defect can't escape twice.
Why it matters
Production defects that testing should have caught — the clearest signal of whether the suite guards the right things.
Flaky-test rate
Measured at work2–3%
Triomics · LiveSwitch
A 2–3% flaky-test rate at Triomics and LiveSwitch, kept there by triaging every failure into product defect vs. automation issue before anyone re-runs. In my public multi-layer framework, any test that only passes on retry is listed in a pull-request comment instead of being hidden by the retry.
Why it matters
Share of tests with inconsistent outcomes — the thing that teaches a team to ignore red builds.
Release readiness
Public project4 gates
This site · every push, plus daily e2e
Every push to this site runs lint, a type-checked build, an axe-core audit of every page, and a Lighthouse accessibility gate (≥95); the full Playwright e2e suite runs on a daily schedule.
Why it matters
Whether a build is safe to ship: defect severity, critical-workflow results, and unresolved risks together.
AI effectiveness
Measured at work70%+
Triomics · LiveSwitch · less time writing & debugging
70%+ less time spent writing and debugging tests at Triomics and LiveSwitch, with every AI-generated test still passing a manual review gate. That's separate from the ~40% regression-time figure, which covers the Playwright migration and AI workflow together.
Why it matters
Time saved by AI-assisted testing, weighed against the review effort it adds and the quality of what it produces.
// case studies
Deep dives
Test strategy, AI-assisted QA, production signals, and accessibility — each grounded in real work or a public repo.
- End-to-End Quality StrategyPersonal or practice project
One order, every layer: a quality strategy for an event-driven app
Product risk, critical workflows, test layers from UI to Kafka, CI tiers, and what the suite actually caught — on a public framework of 154 Playwright tests.
test-layers.json154 tests- API (REST + GraphQL)84
- UI27
- WebSocket11
- Database9
- Kafka4
- End to end2
- AI-Powered QA
Using AI in testing, with judgment attached
Where Claude Code, Cursor, and MCP-assisted workflows speed up my QA work, how I check what they produce, and what I hold them to.
ai-workflow.mdReview-gated- Time writing + debugging tests70%+ less
- AI tests merged without review0
- Assertions from requirements, not outputBy design
- LLM eval cases (DeepEval practice)33
- Production Quality & ObservabilityPersonal or practice project
Turning production signals into test priorities
How error rates, logs, latency, and incidents should feed test prioritization and release decisions — shown on a live app anyone can inspect.
signals.ymlDemo- Server, edge & browser errorsWired
- Source-mapped stack tracesWired
- Rate limits on 5 routesWired
- Structured logs & request IDsNot yet
- Release taggingNot yet
- Accessibility & Risk-Based Testing
What accessibility scans catch, and what they don't
How I audit against WCAG 2.1 and 2.2 A/AA with Playwright and axe-core, and where manual, keyboard, and exploratory testing take over.
a11y-gates.txtWCAG 2.1 / 2.2 AA- Review cadence at Triomics2x/yr → monthly
- Pre-production checklistEvery feature
- Custom checks beyond axe~25
- This site: Lighthouse a11y gate≥ 95
// operating model
Late-Stage QA vs. How I Work
Where the difference shows up, and the real numbers behind it.
| Dimension | Late-stage QA | How I work | Real result |
|---|---|---|---|
| Feedback loop | Regression runs overnight or by hand before a release | A risk-tiered @smoke suite on every pull request, the full suite nightly | 34 smoke tests per PR (public framework) |
| Defect escapes | Found in production and fixed under pressure | Risk-based coverage of critical journeys; every escape becomes a regression test | <5% escaped defects (Triomics · LiveSwitch) |
| Regression time | Long serial runs nobody wants to wait for | Playwright migration, parallel shards, and AI-assisted workflows | ~40% faster regression runs (LiveSwitch) |
| Accessibility | An audit twice a year, when a report is due | A pre-production checklist for every feature, plus a monthly review | 2x/yr → monthly (Triomics) |
| Flaky tests | Re-run until green | Triage product defect vs. automation issue; tests that pass only on retry are surfaced | 2–3% flaky-test rate (Triomics · LiveSwitch) |
// quality engineering
Quality starts at product discovery and continues into production.
Test automation is one tool in that system, not the whole of it. I start in requirement reviews, where an edge case costs a conversation instead of a hotfix, and I stay involved after release, where production incidents tell you what the suite missed.
In between, the work is choosing where to spend effort: risk-based prioritization of customer journeys, shift-left quality, automation where it pays for itself, exploratory testing where scripted checks won't find the problem, and release decisions that run on signals rather than a green checkmark alone.
- Risk-based testing
- Critical customer journeys
- Shift-left quality
- Automation where it pays off
- Exploratory testing
- Signal-driven release decisions
// engineering & recruiter briefing