Analysis

LicenseCustom add-onDetails

Show for

Any licenseInt4 Shield Lite

Feature or limit	Int4 Shield Lite
Test case analysis	Add-on

Anything a license model does not include can be added through a custom license. The license installed for you is shown in the app under Profile → License. All features by license

Overview

Once a regression suite is captured from real traffic it can hold tens or hundreds of thousands of test cases. At that size the useful question is no longer "which test case is this?" but "what business content do my tests cover?" — how many test cases carry each sales organisation, which materials appear in only one test case, which order types are missing altogether. Analysis answers those questions by grouping a chosen set of test cases by the business values found in their messages.

An analysis has two parts:

• A scope — which test cases to look at, defined with the same criteria as a saved search: folders, folder name patterns, search text, automation object type, tags and business dimension values.
• A layout — how to group them: up to six fields as rows, one field as columns, and any number of filters.

The fields you group by are the business dimensions defined on the automation objects of the test cases in scope (for example SALES_ORG, MATERIAL or ORDER_TYPE), plus two attributes of every test case: its Automation object and its AO type. The values come from the extraction that reads each test case's messages; see Business dimensions for how dimensions are defined and extracted.

An analysis is answered from a dataset: a snapshot of the scope taken when you build it. Working with a snapshot is what makes the screen fast and stable — you can reorder rows, add a filter or swap the column field without waiting, and the numbers do not shift while you read them. The flip side is that a dataset is a moment in time: test cases edited, moved, archived or re-extracted after the build are only reflected after you rebuild.

From a built analysis you can:

• read an Overview of the biggest values of the most important dimensions, and how many test cases have no value;
• Explore the scope as a pivot table with a chart, level by level;
• open the test cases behind any count and run them;
• copy or move the selected test cases into a folder tree named after their values — for example one folder per sales organisation with a subfolder per order type.

Tasks

How to create an analysis

1. Open Test Cases → Analysis and click New analysis. As a shortcut, right-click a folder in the Cockpit tree and choose Analyze: the new analysis starts with that folder and its subfolders as the scope.

2. Under Details, enter a Name (unique) and an optional Description.

3. Define the scope with the criteria cards below. All criteria you fill in must match:

   • Folder criteria — add Folder patterns (Include or Exclude, matched against the folder Name or full Path, * matches anything; a matched folder always brings its subfolders), and pick Folders directly, with or without Include subfolders.
   • Test case criteria — search text, an automation object type and tags.
   • Business dimensions — click Add dimension, pick a dimension and one or more values; a test case matches when it has any of the values.

   To reuse criteria you already have, choose a saved search in Start from saved search at the top of the Scope card. Its criteria are copied into the scope; later changes to the saved search do not affect the analysis.

4. Check the notice in the Scope card. It shows how many test cases the scope holds — counting only test cases you may read — and warns when the scope is too large to build or large enough to build in the background.

5. Click Build and open to save, build the dataset and open the analysis, or Save to save without building.

How to build or rebuild the dataset

1. Open the analysis from the list.
2. Click Build (or Rebuild when a dataset already exists).
3. Small scopes are built straight away. Larger scopes build in the background: a progress notice shows the phase (Queued, Freezing test cases, Materialising fields) and a percentage. You can leave the page — you get a notification with Open analysis when the build is ready or has failed. Click Cancel build to stop a build that is queued or running.

A successful build replaces the previous dataset. Rebuild whenever you want the analysis to reflect new, changed or re-extracted test cases.

How to read the overview

The Overview tab opens first and summarises the dataset:

• Test cases in scope — the number of test cases in the dataset that you may read.
• Indexed — how many of them have been extracted for the dimensions shown. A low share means extraction has not caught up (see Tips & edge cases).
• Dimensions and Distinct values — which dimensions the cards below show, and how many distinct values they hold together.

Below the tiles, one card per dimension lists its biggest values as bars, with the number of test cases that have a value and the number without one (hover to see the split into No value, Not defined and Not extracted). Values that only one test case has are highlighted — a quick way to spot thin coverage. The overview shows up to five dimensions: the ones in the analysis's rows and columns first, then the best-covered ones. More dimensions: lists the rest. Cards for Automation object and AO type appear when the scope holds more than one of them.

• Click a value bar to see the test cases that have that value.
• Click Show in Explore on a card, or a dimension under More dimensions:, to put that field first in the rows and switch to Explore.

How to explore by rows, columns and filters

1. Open the Explore tab. The Fields panel on the left lists the Business dimensions and Attributes you can use; type in Search fields to narrow it.
2. Click Add next to a field and choose Add to rows, Add to columns or Add to filters. You can also use Add field and Add filter in the zones above the table.
3. Rows — each row field is one level of the table, in order. Use Move left / Move right to reorder levels; reordering never needs a rebuild. Rows hold up to six fields.
4. Columns — one optional field whose biggest values become columns (up to 20; the rest are summed under Others). Cells are shaded by size, like a heat map.
5. Filters — click a filter chip to edit it. Choose Only these values or All except these values, pick the Values (searchable, biggest first) and click Apply. Remove filter deletes the filter. Several filters must all hold.
6. In the table, the first row All test cases shows the whole scope. Expand a row to break it down by the next row field; long lists load 200 values at a time.
7. When the layout is right, click Save layout so the analysis opens with it next time. The button appears only when the layout differs from the saved one.

The chart below the table follows the level you expanded last. Switch between Bar (the biggest 20 values of the level) and Heatmap (rows against the column field; needs a field in Columns). Click a bar or a heat map cell to see its test cases, and use the level links above the chart to go back up.

How to see and run the test cases behind a number

1. Click a row in the table, a bar or cell in the chart, or a value on an overview card. A window lists the test cases behind that count, as captured at build time, with Number, Description, Automation object, Folder and State.
2. Click a number to open that test case in the Cockpit.
3. To run them, click Execute All to run every test case behind the row, or select some rows first to run only those. Use the arrow next to the button for Execute All w/ Parameters / Execute Selected w/ Parameters to choose the run parameters first. Archived and deleted test cases are skipped and cannot be selected.

How to copy or move test cases into folders by value

Use this to restructure test cases by business content — for example a folder per sales organisation with a subfolder per order type.

1. In Explore, put the fields that should become folder levels into Rows, in the order of the levels you want.
2. Select the rows to transfer with their checkboxes, at any level. A selected row includes everything below it, including values not loaded yet. Active filters still apply.
3. Click Copy structure or Move structure.
4. Pick the Target folder and confirm with Copy or Move.
5. The transfer runs in the background and the window shows its progress. You can close it; a notification tells you when it is finished, with Open folder to jump to the target folder. Cancel transfer stops it before the next batch — what was already copied or moved stays.

How the folders are built:

• One folder level per row field, named after the test case's value (automation objects by name, types by their label). Existing folders with the same name under the same parent are reused; missing ones are created.
• A test case without a value for a row field stops at that level, in a folder named "(no values)".
• Copy creates one copy per folder the test case belongs in: a test case with two materials gets two copies.
• Move puts each test case in exactly one folder, so every selected test case must have only one value per row field. Before you confirm, the window checks this; if some test cases have several values it names the fields and offers Copy instead. A moved test case gets a new version in its new folder.

How to edit or delete an analysis

1. Open the analysis and click Edit to change the name, description or scope. A changed scope takes effect at the next build.
2. To delete it, open the editor and click Delete, then confirm in Delete analysis. The analysis and its dataset are removed. Test cases, folders created by transfers, and any saved search you started from are not affected.

Field reference

Analysis list

Column	Description
Name	The analysis's unique name. Click a row to open the analysis.
Scope	The folders picked in the scope ("(+sub)" when subfolders are included). Shows Saved-search criteria when the scope uses only other criteria.
Layout	The saved row fields, in level order.
Dataset	The latest dataset: its status (Ready, Building, Pending, Failed or Cancelled), the number of test cases you may read and when it was built; the build percentage while building; or Not built.

Fields panel

Element	Description
Business dimensions	One entry per dimension code that at least one automation object in the scope defines together with a way to read its value. The origin reads global for a dimension from the shared catalogue, local for one defined only on automation objects, or "global + N local" when both exist with the same code — they are counted together.
Coverage ("% have a value")	Once a field has been used, the share of test cases that have a value for it.
"not defined in N of M AOs"	How many automation objects in the scope do not define the dimension at all. Their test cases appear as Not defined.
Attributes	Automation object and AO type of each test case — available in every analysis.

Pivot table columns

Column	Description
Value	The value of the row's field. A warning icon marks a row whose children add up to more than the row itself (see Tips & edge cases).
Test cases	Distinct test cases with this value, within the parent row and the filters.
Exclusive	Test cases whose only value for this field is this one.
% parent	Share of the parent row's test cases.
% scope	Share of all test cases in scope (after filters). Shown when no column field is set.
Column values and Others	With a column field: the row's test cases that have each of the 20 biggest column values, and those with any other value.

Values without data

Every test case is counted for every field. When it has no value, it is shown under one of these entries instead, so nothing silently drops out of the totals:

Entry	Meaning
(Not defined)	The test case's automation object does not define this dimension, so there is nothing to look for.
(Not extracted)	The test case has not been extracted yet, or was extracted before the automation object's dimension definitions last changed. A rebuild after extraction has caught up fills these in.
(No value)	The test case was extracted and its messages hold no value for this dimension.

Test case states in the drill-down

State	Meaning
Current	Unchanged since the build.
Newer version since build	The test case was changed after the build; the counts reflect the version captured at build time.
Archived	Archived after the build. Cannot be run or transferred.
Deleted	Deleted after the build. Cannot be opened, run or transferred.

Tips & edge cases

• Child rows can add up to more than their parent. A test case can carry several values for one dimension — an order with three items has three materials. It is then counted once under each value, so the children of a row can sum to more than the row. The table marks such rows with a warning icon and explains it in a notice. Test cases is always a count of distinct test cases, and Exclusive tells you how many have only that one value.
• Many "Not extracted" entries? Values come from the extraction that runs when test cases are captured or changed, and again after an automation object's dimension definitions change. Until it has caught up, test cases appear as Not extracted and the Indexed tile is below 100 %. Wait for extraction to finish, then rebuild.
• Everyone sees only what they may read. The build takes only test cases the builder may read, and every viewer then sees only the test cases in folders they may read — counts, values and totals included. Two colleagues can therefore see different numbers for the same analysis. Analyses themselves are not limited by project scope: everyone who may use the feature sees the list of analyses.
• Scope size limits. A scope can hold up to one million test cases. Scopes above 25,000 test cases build in the background; the editor tells you which applies before you build. Your administrator can change both limits.
• Slow questions are stopped. A table, chart or filter request that takes longer than 10 seconds is stopped with a message asking you to add filters or use fewer rows. Add filters or expand fewer levels at once.
• A failed or cancelled rebuild hides the previous results. The analysis always shows its newest dataset. If that build failed or was cancelled, run Rebuild again to get a usable dataset.
• Only one build and one transfer at a time per analysis. A second build or transfer of the same analysis is refused until the first has finished.
• Building does not save the layout. Rows, columns and filters are kept only when you click Save layout.
• Transfers skip what you may not change. Test cases are skipped (and counted as skipped in the result) when they were deleted or archived since the build, when you may not create test cases or folders in the target, when you may not read a copied test case's automation object, when your licence does not allow creating the copy, or — for a move — when you may not change test cases in the source folder.
• Scope criteria the editor cannot show. A saved search can contain criteria that the analysis editor does not display, such as a list of automation objects. Those criteria are not carried into the analysis when you start from such a saved search.

Related features

• Business dimensions — define the dimensions an analysis groups by.
• Automation objects — where dimensions are attached to an interface and told where to find their value.
• Cockpit — the folder tree an analysis scopes and restructures; right-click a folder and choose Analyze to start an analysis from it.
• Crawlers — capture test cases in bulk, the typical source of a large scope.
• All test runs — follow the runs you start from a drill-down.

Prerequisites & permissions

About permissions. What you can see and do is decided by the roles assigned to your user. Each role bundles permission groups, and each permission group allows activities — Read, Write, Delete, Execute and, for schedules, Change owner — on a kind of record (for example Test Case, Automation Object, System Under Test). A role grants its groups per project scope, so you may be allowed to change records in one project but only view, or not see, those of another.

If a button is disabled, a page says you don't have access, or a list comes back empty, missing permissions are the most likely cause — or, for whole product areas, the installed licence. You can review your own access under Profile → My access; an administrator can change your roles under IAM → Users.

Analysis requires the test case analysis feature in your licence; without it the menu entry is hidden. Within the feature:

• Test Case Analysis: Read — open the list and analyses, use the overview, the pivot, the charts and the drill-down. You also need Test Case: Read in the folders whose test cases you want to see.
• Test Case Analysis: Write — create analyses (New analysis, and Analyze in the Cockpit), edit them and save layouts.
• Test Case Analysis: Execute — build, rebuild and cancel builds, select pivot rows and copy or move structure. Copying or moving also needs Folder: Write and Test Case: Write in the target folder, and the test case management feature in your licence.
• Test Case Analysis: Delete — delete an analysis.
• Test Case: Execute — run test cases from a drill-down.

Was this page helpful?Send a quick note to the documentation team.

0 / 2000 characters