Eduardo Ramos

About

Expertise

Experience

1. roadsurfer

   Remote, Spain

   QA Test Automation Engineer 2025-Present

   • Design and maintain UI automation in Playwright with Java and API tests in REST Assured inside a central Gradle multi-module framework (Spring Boot, JUnit 5, Allure), covering booking, back-office operations, user accounts, subscriptions and sales for a platform running in around 100 locations across 16 countries; also maintain legacy Selenium suites.
   • Built a reusable determinism test module for generative-AI features whose outputs cannot be exact-matched: it replays fixture scenarios through an AI classification workflow, scores structured and free-text responses with an LLM-as-judge against per-fixture criteria, and records per-field drift rate, enum distribution and judge pass rate per build, tagged by model and workflow version so a regression can be attributed to a model or a prompt change.
   • Own nightly Jenkins regression triage: reproduce against staging and feature environments, separate real product defects from test decay, raise and classify defects in JIRA, validate redesigned flows before release, and gate coverage for unreleased features instead of leaving red builds.
   • Validate behaviour across UI, API and database together, combining REST Assured helpers against backend services with SQL queries confirming persisted state matches what the UI shows, including flows that carry customer personal data.
   • Improved failure diagnosis so debugging is not archaeology: Playwright traces captured into Allure for inline, shareable reports. Write Given/When/Then specifications for stakeholders, review teammates' pull requests, and author the internal skills that standardise handovers, pull requests and code review across QA.

2. Fleksy (keyboard SDK)

   Remote, Spain

   QA Engineer 2022-2025

   • Designed, developed and maintained automated UI and API test suites with Selenium in Java, Kotlin and Python for new and existing SDK features.
   • Built workflows and suites for smoke, sanity, functional, regression and usability testing across browsers and on iOS and Android devices, analysing user stories and requirements to find quality risks ahead of automation.
   • Prepared and maintained test cases from user stories throughout development, wrote detailed test summary reports, and coordinated with stakeholders on testability and test infrastructure.

3. Interaction Design Foundation

   Remote

   Software Tester, Product Operations 2018-2022

   • Led software testing strategy for an online learning platform used by 1,576 companies with over 1.1 million course enrolments, establishing the testing practice for a product team with no dedicated QA function.
   • Mentored staff day to day, turned product feedback into actionable items for the development team, and ran functional and usability testing across member-facing and internal systems.

4. Sitio Uno

   Caracas, Venezuela (remote from 2018)

   QA Manager (2013-2018), then QA Consultant 2013-2021

   • Led certification of payment software with FirstData/Fiserv over roughly two years, covering EMV payment technology and the PCI PA-DSS security standard across combined hardware and software, and handled version control of the application, its releases and the test evidence submitted for certification.
   • Directed two technical writers through the 20+ implementation guides and certification documents the standard required, and built the test plans, product requirements and customer support plans for an agile team from scratch.
   • Led QA for payment software, point-of-sale hardware and payment reporting applications; two EMV projects reached over 100,000 users in their first two years. Retained part-time as QA Consultant from 2018, advising on the platform and its certification work until November 2021.

Selected work

Three decisions from qa-automation-framework, a skeleton for wiring AI agents into a real test automation stack, web and native mobile, stack-agnostic and driven by one config file and per-stack profiles. Each page is one decision, what it found, and what it cost. Every number comes from that repository.

• One file an agent reads, one file a stack owns

  The AI layer knows QA and nothing about Playwright or pytest. The profile layer knows one stack and no QA reasoning. qa.config.yml plus each profile's profile.yaml is the only contract between them, so adding a language means writing files, not rewriting agents.

  4 files, 0 agents changed to add a stack

• A policy hook that only judges the lines you touched

  Forbidden patterns are checked on every write, but only against lines the working tree actually changed, diffed against git show HEAD. Dropping the hook into a repo with two hundred pre-existing violations does not fail every edit to a file that merely contains one.

  0 pre-existing violations ever block a write

• One ticket, five workflows, three defects and one cause

  A single ticket run through refinement, design, generation, exploratory testing and audit. The exploratory session found three defects that all trace back to one thing: the cart is never re-read at confirm time. The audit then found the highest-risk row in the whole design has no test anywhere.

  3 defects, one root cause

Two more decisions in that repository that did not get their own page: the profile-writing workflow describes a repo's existing test style rather than correcting it, so what a suite does and what it should do never end up mixed in the same file; and the same five workflows run unchanged against three different stacks in the demos, TypeScript, Python and mobile, with no agent needing to know which one it is talking to.

AI determinism-checker

Catching when the model quietly starts giving worse answers

A model call can be subtly wrong in a way an ordinary pass or fail test never catches. With this in mind, I built the ai-determinism-checker. This suite runs the AI many times against a fixed input and checks every field against a rule, so drift turns up in a nightly run instead of in production.

You cannot exact-match a model

Ask a regression test whether the output equalled X and a model breaks it on day two. The wording shifts, a part id changes, an enum drifts. The right question is not does it equal X but is every field still within what is allowed. And because a model can be inconsistent, you have to ask more than once.

One fixed input, N runs, every field checked

1. Fixed input Report text A canned description plus the ids it needs.
2. Run ×N Feature webhook The API test posts the same body N times, no browser involved.
3. Collect N results Each one a full field set.
4. Evaluate Rules Each field against the rule bound to it.
5. Report Tick or cross per rule Actual against allowed.

Every run is checked on its own, so drift on any single run fails the case.

Four kinds of check cover every field

The engine does not know which feature it is testing. Each field is bound to one of these.

• Allowed set The value must be one of a fixed enum, or of a list exported from the data the feature reads. category in {SCRATCH, DENT, ...}
• Allowed range A numeric value must fall inside bounds. The engine provides it; no numeric field uses it yet. 0.0 <= x <= 1.0
• Cross-field A relationship between fields on the same result has to hold. These cost nothing to run and catch the worst failure, which is the model inventing certainty it does not have. empty part_id => follow_up set
• LLM judge Free text goes to a separate model that returns pass or fail against plain-language criteria. description reads as a real, laconic note

Free text is graded by its own workflow

A description cannot be checked against a list, so it goes to a small judge workflow that QA owns: a webhook in, a model call, a structured verdict out. It sits behind an interface, so no model credential lives in the test code and an offline stub replaces it entirely.

1. Send { content, criteria } the field text, plus what “good” means for this fixture
2. Judge the workflow calls the model at temperature 0, pinned, forced to JSON
3. Verdict { pass, reason } a false is a real finding, and the reason says why
4. Fail loud a broken judge is a hard test failure, never a fabricated false. An error and a disagreement are different things.

What it caught on the first run

Four fixtures ran. Three passed. One failed, correctly.

A true positive. The classifier invented detail and editorialised, against the rules in its own prompt. Nothing in an ordinary test would have flagged it, because the flow worked and the payload was well formed. What the suite caught was the drop in quality.

The engine underneath, meaning the repeat runner, the rules and the reporting, has no knowledge of the feature and no production dependencies at all. The next model-backed feature plugs in its own result type and fixtures rather than a new framework. It runs nightly, never blocks a pull request, and prints actual against allowed on every failure, so the ticket writes itself.

Read the full write-up

QA Lab

This site is plain HTML, CSS and JavaScript with no build step. It has a Playwright suite that tests it, and that suite runs on every push and pull request.

Contact

LinkedIn is the fastest way to reach me. I keep no email address on this page.

• LinkedIn eduardo-ramos-anderson
• GitHub eduramos21