Test discovery and tags

Organize tests with tags and control which flows run using include and exclude filters.

As your test suite grows from a few files to hundreds of Flows, managing which tests to run and where they are located becomes critical. Astra Scout uses a combination of inclusion patterns and metadata tagging to give you precise control over your execution.

Test discovery via patterns

By default, when you run a whole workspace folder, Astra Scout only executes flow files located at the top level of that directory. It ignores subfolders to prevent accidental execution of nested or utility Flows.

To change this behavior and organize your repository into subdirectories (like /auth, /checkout, or /journeys), you must define Inclusion Patterns in your config.yaml.

The flows key uses glob syntax to define which files should be included in a test suite run.

# config.yaml
appId: com.example.app
flows:
  - "*"            # Default: Includes all Flows in the root folder only
  - "auth/*"       # Includes all Flows in the auth subfolder
  - "tests/**"     # Recursive: Includes all Flows in tests and its subfolders

Exclude files with negation patterns

Prefix a glob with ! to exclude files that would otherwise be included. Use this to skip anything you don't want Astra Scout to run, whether that's a .yaml file that isn't a Flow (such as a helper or data file living alongside your tests), a work-in-progress Flow, or an entire subdirectory.

Astra Scout first collects every file matched by the positive patterns, then removes any file matched by a negation pattern.

# config.yaml
appId: com.example.app
flows:
  - "**"                    # Include every Flow, recursively
  - "!ignored.yaml"         # Exclude a single file
  - "!config_dir/**"        # Exclude an entire subdirectory
  - "!featureA/skip.yaml"   # Exclude a specific file within an included folder

WARNING

A flows list needs at least one positive pattern. A list made up only of negation patterns matches nothing, so no flow runs.

Filter test via tags

Tags allow you to categorize your Flows based on their purpose, priority, or execution environment without changing your folder structure.

To add tags to a Flow, you need to define them in the configuration block at the top of your individual Flow files:

# flows/login_smoke.yaml
appId: com.example.app
tags:
  - smoke
  - registration
---
- launchApp
# ... rest of the flow

When you run a workspace, Astra Scout filters the flows by the tags of the environment you selected. In the Environments sheet (Run → Manage Environments…), each environment has an Include Tags field and an Exclude Tags field that take comma-separated tags.

• Include: put smoke in Include Tags to run only smoke tests.
• Exclude: put wip in Exclude Tags to run everything except work-in-progress tests.

Tag logic

When you list several tags in one field, the filter uses OR logic.

• Multiple includes: auth,checkout runs any flow tagged auth OR checkout.
• Multiple excludes: experimental,stagingOnly skips any flow tagged experimental OR stagingOnly.

AND operator

There is no option to run only flows that match a combination of multiple tags (AND logic) in one field, for example only flows tagged both smoke and auth.

If you use both include and exclude tags together, they perform an AND operation between the two groups:

1. Astra Scout identifies all flows containing the included tags.
2. It then removes any flows that contain the excluded tags.

Consider you have two Flows:

• Flow A: tags: [dev, pull-request]
• Flow B: tags: [dev]

Flow A

# flowA.yaml
appId: com.example.app
tags: 
  - dev
  - pull-request

Flow B

# flowB.yaml
appId: com.example.app
tags: 
  - dev

Include Tags	Exclude Tags	Result	Why?
dev		Flow A & B	Both contain the dev tag.
dev,pull-request		Flow A & B	Uses OR logic, both contain at least one of the tags.
	pull-request	Flow B	Flow A is removed due to the pull-request tag.
dev	pull-request	Flow B	Both are included via dev, then Flow A is excluded.
	dev	No flows run	Both flows contain dev and are removed.

Global tag configuration

While individual tags in a Flow file categorize what a test is, defining tags in your config.yaml sets the policy for your entire workspace. It acts as a permanent gate, ensuring that specific types of tests are always included or ignored without you having to remember to fill in the environment every time.

# config.yaml
includeTags:
  - production_ready
excludeTags:
  - experimental
  - flaky

You can use global tags to:

• Target environment: Use an environment with its own include and exclude tags for each stage, for example one that includes tags for internal-only features that shouldn't run in production.
• Enforce quality gates: If you have 10 tests that are currently "flaky" due to a known environment bug, you don't want to delete them. By tagging them as flaky in the Flow and adding excludeTags: [flaky] to your config.yaml, you ensure no one on the team accidentally runs them and breaks the build.

INFO

The tags from the selected environment and from config.yaml are combined: the include lists are merged, and so are the exclude lists. A flow runs when it matches an include tag (if any exist) and no exclude tag.

Example

Imagine a repository structured by feature:

.
├── config.yaml
├── auth/
│   ├── login.yaml (tags: [smoke])
│   └── signup.yaml
└── payments/
    ├── credit_card.yaml (tags: [smoke])
    └── paypal.yaml

To run your smoke suite across all features, your config.yaml should look like this:

# config.yaml
appId: com.example.app
flows:
  - "**"          # Look everywhere
includeTags:
  - smoke         # But only run the smoke tests

Next steps

Now that you've discovered your tests, learn how to manage their sequential execution.