One file an agent reads, one file a stack owns
- Project
- qa-automation-framework
- Component
- Config contract,
qa.config.ymlplus per-profileprofile.yaml - Stack
- Python, stdlib only, ships as a Claude Code plugin
The problem
An agent that can reason about test design still has to run a suite it has never seen, in whatever language, runner and CI a project already picked. Teach that directly to the agent and QA reasoning ends up entangled with the mechanics of one runner, so the next stack means editing the agent that was supposed to be stack-agnostic in the first place.
That coupling is the failure mode worth designing against: an agent that only works on the stack it was written against is not a framework, it is a script with a language model attached.
What I built
Two layers, and exactly one contract between them. agents/ and
skills/ know QA and know nothing about Playwright, pytest or Appium.
profiles/<name>/ knows how to run one stack, where its files go and
what good code looks like there, and holds no QA reasoning at all.
| Layer | Knows | Does not know |
|---|---|---|
agents/, skills/ | Requirement refinement, risk-scored test design, exploratory charters, audit rubric | Playwright, pytest, Appium, any specific runner |
profiles/<name>/ | Run and test-one commands, file layout, selector priority, house conventions | Whether a test is worth writing, what to check, how to score risk |
The contract is qa.config.yml at the project root, which picks a profile per
stack, plus that profile's own profile.yaml. When an agent needs to rerun one
test it reads commands.test_one from the profile. When it needs to write a
page object it reads the profile's template and its CONVENTIONS.md. Neither
file exists inside the agent.
What it verifies
qa-context web resolves the whole thing in one command: config path, which
profile the stack points at, and the profile directory, so a hand-off to an agent is one
command instead of a paragraph of instructions somebody has to keep current.
The claim is checked, not assumed. scripts/validate_profiles.py runs in CI
against every shipped profile, and the repo carries three of them plus a
_template: ts-playwright, py-pytest-playwright and
mobile-appium-wdio. Three different languages, three different runners, and
the agents in agents/ and the workflows in skills/ do not
change between any of them.
What it cost
Adding a fourth stack is copying profiles/_template, filling in
profile.yaml, writing a real CONVENTIONS.md with examples lifted
from actual code, and running the validator.
Four files, zero agent changes. Java plus Selenium plus JUnit 5 is
deliberately not one of the three shipped profiles: it is the obvious next one to write,
specifically because writing it from the template is the fastest way to find out whether
the contract actually holds, rather than only looking like it does on the stacks it was
designed against.