Versions Compared

Old Version 1

changes.mady.by.user Unknown User (lfir)

Saved on

compared with

New Version 2

changes.mady.by.user Unknown User (lfir)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
UI Steps
UI Step
Historical Requirement

Coverage Report

Image Removed

UI Step

Purpose

This report allows users to evaluate the progress of the requirement coverage status, which implicitly takes into account the testing results associated with each requirement.

Shows the evolution of the calculated requirement coverage status for a list of requirements, over a time range, in stacked bar charts,

Calculates the test coverage status for a list of coverable issues for the selected version

,

or Test Plan, and/or Test Environment.


This helps you analyze the progress of the requirement coverage status, by

The statuses are presented in a normalized horizontal stacked bar chart and  evaluates the completeness of each issue.


Image Added


UI Step

Purpose

In order to analyze the current status of your release, or even past releases, you need to evaluate the status of the requirements and other covered issues, taking into account the

testing results associated with each requirement until the day being analyzed.

respective results.

You might want to analyze just the requirements implemented

Possible usage scenarios:

Analyse requirement status

in a given

time or in a given time frame
  • Analyse completion of tests
  • Acknowledge the evolution of testing progress (within a given day, within a week, within a month)

    • version or you may want to analyze requirements from earlier versions based on your regression testing.

    The Test Coverage report provides the means to analyze your requirements statuses in real-time.

    Get insights of your performance, and help estimate if you are meeting the deadline for your releases


    UI Step

    How to use

    This report is accessible either accessible from the Xray Reports icon  icon on the project left sidebar of the project or from the standard Reports icon, which includes other kinds kind of reports besides Xray. 

    At the top of the report you'll find several areas related with the report and with the data shown in the report.

    Image Removed

  • A: Analysis & Scope, for choosing how to analyze the entities
  • B: Group by, for defining how to visually group/aggregate items
  • C: Filter, for selecting the source data
  • D: Data range
    • E: Visualization information and options

    You need to configure the source data (i.e., the requirements) for the chart and how to calculate the requirement status for each. 

    , you have some options that need to be configured.

    • Analysis & Scope: how to analyze the issues (more info on Coverage Analysis)
    • Group By: how to visually group the coverable issues (e.g. by prioriy, by component)
    • View: flat ot hierarchical
    • Filters: the source data for the report (e.g. the requirements or other covered issues) and how to evaluate the status for each one of the item being shown. 


    Image Added


    UI Step

    Source data / Filters

     The source data for the report (e.g., the requirements or other covered issues) can be defined under Filters, which available at the top left-hand corner. 

    Image Added

    Source "requirement"/coverable issues can be directly provided using a Basic or Advanced filter:

    • Advanced: a JQL query to filter the coverable issues
    • Basic:
      • Project: project
      • Assignee: the issue's assignee
      • Component: the component assigned to the issue
      • Fix Versions: versions assigned to the issue
      • Resolution: the workflow resolution
      • Status: the workflow status
      • Contains: the full issue key or part of the summary
    UI Step

    Analysis & Scope

    On the left side, you can define the 

    UI Step

    Defining Analysis Strategy

    On the left side (within section A), it's possible to define the analysis strategy, i.e., the way you want to analyze the selected/filtered requirements. You can choose either to analyze either by by Version or  or Test Plan, and then complement it with a Test  Test Environment.


    If you choose analysis by by Version, then the latest results from Test Executions for the specified version are taken into account. If you choose choose Test Plan, only Test Executions (and related results) associated with the given Test Plan are considered in during the calculation of the requirement status.

    If you specify the Test Environment is specified, then only the executions that ran in the specified environment are considered. 

    Image Removed

    To analyze the daily coverage of requirements in a specific date range, you must select the time period (within section D). 

    Image Removed



    Image Added

    Info

    Learn more

    Please read Coverage Analysis for a more detailed explanation of how to perform coverage analysis.

    Defining Scope with Basic Tab

    Source "requirement" issues can be directly provided (within section C) using the requirement fields configured in the Basic tab or the JQL written in the Advanced tab.

    By default, the Basic tab provides the following fields: 

    • Fix Version: Version assigned to requirements
    • Sprint (Only if Jira Software is installed): Sprint assigned to requirements
    • Component: Components assigned to requirements
    • Status: The Issue's workflow status
    • Resolution: The Issue's workflow resolution
    • Contains text: Filter requirements by text

    Image Removed

    By clicking on More, it's possible to manage which fields will be used to filter the requirements:

    • Selecting fields will enable to filter further the requirements
    • Unselecting fields will remove them from the search criteria
    Image RemovedImage Removed
    UI Step
    Info

    The default fields cannot be removed from the dialog

    Group By

    You can visually group the requirements by Priority, Component or other fields, so you can analyze requirements from different perspectives.


    Image AddedImage Added


    Info

    Tips

    For more information, please see the Coverage Analysis.



    Defining Scope with Advanced Tab

    Alternatively to the Basic, the Advanced tab offers the possibility of filtering the requirements via JQL:

    Image RemovedImage Removed

    If you wish, you can clear the filter in order to see all Requirements once again. You can do this by clicking on the Clear button and then press Apply.

    UI Step

    View

    You can see a flattened list of coverable issues or just the top-level ones (e.g., the top-level parent requirements).

    For example, if you're using Epics and Stories and you choose the "Flat" view, then only the Epics will appear in the report, independent of whether the source data includes the sub-issues or not. If you choose the "Hierarchical" view, then only the Epics would be shown. 

    • Hierarchical - only the parent coverable issues will be presented in the chart. Child issues
    UI Step
    UI Step

    Select Visualization Options

    There is also an options menu (within section E) where you can choose the visualization type for the report:  

    hierarchical - only the parent requirement issues will be presented in the report, if the filter includes them. Sub-requirements
    • can still be visualized in the overall details table.
    flatten
    • Flat - 
    the requirement issues will not consider
    • the parent/child relationship between coverable issues is not considered. All parent and child
    requirements
    • issues will be considered
    and showed similarly in the report
    • for the chart. 

    Image RemovedImage Added



    Charts are interactive. Hovering over the different bars

    shows a small popup with the information

    about

    on the specific series (

    Status~Grouping

    Status,

    Date

    Grouping Field and Percentage

    /Value

    ).

     

    You

    can

    can  deactivate a particular series by clicking it in the series legend

    Image Added

    The chart also supports drill-down.

    UI Step

    Example

    This graphical report is composed of two similar vertical stacked bar charts: one showing absolute values and the other showing relative percentage normalized values. 

    Each chart column shows the Requirement coverage status for a given day. Each status has a different color and is divided by Requirement Grouping with a gradient.

    Image Removed

    Each bar on the chart represents a group of Requirements with a different value provided by the grouping field.  Stacked within each bar are the Requirements grouped by coverage status: OK , NOK , NOT RUN , UNKNOWN and UNCOVERED .  

    Because this is a normalized chart, the bars are always 100% wide.

    The Overall Coverage requirements chart is interactive. Hovering over each bar

    Tip
    UI Step

    When a section is selected, a table appears below the chart with the Requirement issues.  

    For each Requirement issue, the table shows the following indicators:

    • Total Tests: total number of linked tests
    • Passed: total number of tests passing (i.e., the ones whose status is mapped to the Test Coverage status "OK") 
    • Failed: total number of tests failing (i.e., the ones whose status is mapped to the Test Coverage status "NOK") 
    • Other: total number of tests whose current status is mapped to the Test Coverage statuses "Not Run" or "Unknown"

    It also contains a column with a progress bar that shows the completeness of each issue.

    Some values may provide a link, so you can be redirected to the respective entities corresponding to the number shown.

    Image Added

    Sharing the Report

    The report can be shared by copying and sending the URL located in the browser's address bar.

    When opening the report, the Analysis & Scope, Filter, Group by, Date range and Requirement Presentation options will be automatically populated with the values provided in the URL.