QA Studio is a reference solution built with Amazon Nova Act. Part 1 introduced test definition in natural language and AI-powered visual navigation. Part 2 added test suites and a command-line interface for batch regression testing in CI/CD pipelines. Part 3 addresses the organizational and operational layer around those tests: grouping them by application, turning execution outcomes into health information, and controlling who can see or change each resource.

Applications define the system under test

In QA Studio, an application represents a system under test. It is also the container and ownership boundary for that system’s use cases and test suites. This gives related tests a shared context and places their organization, reporting, and access rules beneath a defined owner.

Creating an application requires a name and a base URL. It can also include an optional set of environments, such as development, staging, and production. The environment model lets execution information remain associated with one application while still distinguishing where a test ran.

Execution outcomes become an application health view

QA Studio records every execution outcome as an Amazon CloudWatch custom metric. The metrics are dimensioned by application and environment, and QA Studio uses them to construct an application health view. Individual results therefore contribute to an aggregate picture tied to both the tested system and the relevant environment.

The application page reports the pass rate for the selected time window, the total number of executions, and the current failure count. It also presents a health score from 0 to 100. That score is calculated from the pass rate and includes a penalty for flaky tests. The summary consequently reflects both overall success and instability in test outcomes.

The metrics tab supplies two time ranges with different levels of detail. Its seven-day view has hourly granularity, while its 30-day view has daily granularity. The charts cover executions, successes, failures, and average duration. An environment filter makes it possible to examine those measures for the selected application context.

Finding unstable tests and investigating failures

A dedicated flaky-tests view ranks use cases according to their pass-to-fail and fail-to-pass transitions during the previous seven days. It focuses attention on tests whose outcomes have changed rather than treating every recent result as equivalent. A separate recent-failures view provides entries that link to Amazon Nova Act trajectory logs and session recordings, connecting the overview to records for a particular failed execution.

Teams establish ownership and visibility

Access in QA Studio is organized around teams. Every application belongs to exactly one team, and members of that team can work with it according to their roles. Users outside the owning team do not see the application unless access has been shared. Ownership is therefore the default visibility boundary as well as the basis for managing work on the application.

QA Studio defines five cumulative roles. An Admin has studio-wide access. A Team Lead manages team members and creates applications for the team. An Editor creates and edits use cases and test suites. An Executor can run them but cannot modify them. A Viewer has read-only access. Because the roles are cumulative, each higher role includes the scope available to the roles below it.

The owning team can share an application with another team at one of three levels: view, execute, or full access. Effective permission is the lower of the recipient’s own role and the access level granted through sharing. A recipient therefore cannot use a share to exceed the capabilities of their role, while a powerful role does not override the limit chosen by the application’s owner.

This combination separates two questions. The share determines how much access the owning team offers to the receiving team, and the recipient’s role determines how much that individual is otherwise allowed to do. The lower-of-two rule resolves the result without discarding either boundary.

How the access model is implemented

The implementation maps each team to Amazon Cognito groups for its roles. Amazon DynamoDB stores teams, memberships, application ownership, and share grants. AWS Lambda and Amazon API Gateway enforce role scope and resource access checks. The visible role and sharing model is consequently backed by checks involving both a user’s team role and the resource being requested.

Applications, metrics, and team-based access control are all included in the QA Studio reference solution. It is deployed with AWS CDK in the user’s AWS account. Test data, results, and metrics remain within the boundaries of that account, placing the operational records alongside the user’s deployment of the solution.

Part 3 therefore extends QA Studio beyond defining tests and running regression batches. Applications provide an ownership and organizational boundary; CloudWatch metrics turn executions into trend and health views; and teams, roles, and sharing rules define visibility and permitted actions. Together, these elements supply the management context around the natural-language tests, visual navigation, test suites, and CI/CD-oriented CLI introduced in the earlier parts.