Attorney Share
Legal marketplace
~500
E2E tests
across ~25 suites
94.3%
pass rate
42,789 runs
~2×
faster runtime
~24 → ~12 min
80
flaky tests
surfaced & tracked
Narrative format
I'm the only QA engineer on Attorney Share. At this size, quality can't depend on how fast one person types — it has to come from a system. So instead of hand-maintaining a pile of tests, I build and run a test system, and I use Claude Code to do it. I plan the work and review every change; Claude writes most of the code and debugs from the logs. Everything still ships under my name, and one person ends up keeping pace with a team.
This stopped being pair programming a while ago. I give Claude a goal and the limits it has to stay inside, and it plans the steps and works through them on its own until the goal is reached. The autonomy only works because the guardrails are real: every effort starts from a written plan I approve, work comes back as small pull requests I read before anything merges, and the end-to-end suite decides whether a change is correct. When something breaks, Claude reads the logs and works back to the cause — one CI failure had been failing quietly for over a week before Claude traced the root cause and added a guard so it can't fail silently again.
Every ticket gets read, challenged, then verified. When a developer marks a ticket ready for QA, Claude reads the ticket and the pull request behind it, then runs specialized review agents over the change — each looking from a different angle: whether the logic holds, what the edge cases are, where it would break. That pass alone has caught things a green checkmark hides, like an 'all tests pass' claim that wasn't true. Only then does it run the E2E suite against the right preview environment, tell real product bugs from test bugs, and post the verdict back on the PR and ticket.
Built from scratch: there was no Playwright framework before this engagement — page models, fixtures, auth and CI against live previews, now ~500 tests across ~25 areas. Accelerated: two careful refactors roughly halved runtime, from ~24 to ~12 minutes. Made intelligent: Smart Test Selection recommends only the tests a diff could affect, running in shadow mode behind drift detection, confidence thresholds and an auto-revert. Made measurable: Exolar reports failures, stack traces and flaky tests, and exposes an MCP so Claude can triage a run itself — real product bug, flaky test, or environment problem.
The cheap inference that runs on every pull request inside Smart Test Selection uses a small, low-cost model that isn't Claude — I picked it because it runs constantly and has to be cheap. Claude is the engineer behind the actual system: it designed the architecture and wrote the framework, the analytics and the safeguards, and I reviewed and approved every change before it shipped. What Claude gives me is steady engineering work over months, not one clever API call. One QA engineer, keeping pace with a team.
Get a senior engineering lead on a call and a concrete plan in days, not months.