Page History

...

Everytime a Test is executed within some Test Execution context, a Test Run is created .containing a copy of the test specification along with the recorded results.

One of the key features of Xray is its coverage analysis capability, which is something quite powerful that most tools are unable to provide. In brief words, Xray provides the ability for you to evaluate in real-time how a given Test or a given "requirement" are for a specific context (i.e. version, version+environment); Xray calculates this based on the latest consolidated results for each context. This means that you may evaluate how a Test or a requirement are for different versions, for example, because for each case it will only consider the testing results obtained for that version.

...

1.  Review the "Max results per request" setting in the the Miscellaneous administration settings as it controls the pagination on the REST API calls. The default value should be ok.
2. Image Removed export results - custom fields
3. https://confluence.xpand-it.com/display/XRAY/Export+Execution+Results+-+REST
4. Diamantino Campos  vai ver o q mandou pra ubs
5.  Limit API calls (to Jira and Xray related onesendpoints) using a reverse proxy
   1. Evaluate what REST API calls are being used, discuss their real need with users
      1. Make sure that pagination is being used on the REST API calls
   2. Restrict access to REST API calls
      1. Limit access to well-known hosts/applications

Reporting

JQL

Xray provides dozens of JQL functions but you have to use them carefuly to make sure your instance is not affected. Please do train your users on JQL before "allowing" to use them throughout Jira.

...

1. Image Added Export results endpoint (i.e. /rest/raven/1.0/testruns) allows you to include custom fields from the Test issues in the response, using includeTestFields parameters; please choose carefully what fields you choose to include, as some of these may be calculated and thus add some additional overhead to the request
2. Whenever searching for issues using Jira's REST API (i.e. api/2/search), please choose explicitly what fields to return iusing the fields parameter; that will avoid including unnecessary fields that are included by default and that can add 

3. api/2/search

    
4. https://confluence.xpand-it.com/display/XRAY/Export+Execution+Results+-+REST
5. Diamantino Campos  vai ver o q mandou pra ubs

Reporting

JQL

Xray provides dozens of JQL functions but you have to use them carefuly to make sure your instance is not affected. Please do train your users on JQL before "allowing" to use them throughout Jira.

1.  Image Added Using unoptimized JQL queries can degrade performance substantially
   1. most times this happens because users don't understand how JQL works first of all; JQL is not like SQL (see Understanding JQL Performance). Thus, filtering issues by project by adding the "project = <xxx>" clause is not the same as specifying the project as argument to the subsequent JQL function.
      
      1. Example: 

         1. Use

            issue in requirements('OK','CALC')

            ...instead of ...

            project = 'CALC' and issue in requirements('OK')
            
            

2. Image Added Some JQL functions, such as the ones dealing with requirement coverage, may be more intensive than other ones, since Xray may have to, for example, load all the related Test Runs in order to obtain relevant data. The following JQL 

   1. testPlanTests() - whenever by tests in a given status; the current workaround is to search using the "TestRunStatus" custom field. 

      Note
      title Please note
      When searching for Tests with a certain status inside a Test Plan, we recommend you to use the custom field search instead. Xray has created a new way of searching with big improvements when filtering by test status, using the Custom Fields: (3) issuetype = Test and TestRunStatus = "DEMO-10 - TODO" (4) issuetype = Test and TestRunStatus = "DEMO-10 - TODO environment:IOS"

   2. requirements() - whenever filtering by by dates
   3. testExecutionTests() (Diamantino Campos tem duvidas... podes confirmar pf?)  - whenever filtering by tests/requirements in a given status; it will depend on the amount of Tests you have on the Test Execution
   4. parentRequirements() (Diamantino Campos tem duvidas.. podes confirmar pf?)  - depending on the amount of requirements and sub-requirements you have, it can take a while to complete and require some resources
   5. Diamantino Campos , any other Jql function?

Xray calculated custom fields

1. Image Added  Xray provides some specific custom fields that calculate their values on the fly. This means that you should have that in mind, specially if you're including them in tables/issue listings/gadgets.
   1. The most intensive custom fields are the "Test Execution Defects" and the "Test Set Status". The "Tests Count", as it does an aggregation, also is intensive if you use it for multiple issues.

Reports and Gadgets

1. Image Added One way of doing reporting is by using gadgets. Gadgets are great to share information between team members and even between different teams; however, if not use carefully, they can degrade JIRA performance if all users have the same report on their dashboard as they will probably generate multiple requests once users access the dashboard. Thus, use carefully most intensive gadgets such as the "Historical Daily Requirement Coverage" and the "Tests Evolution" gadget and others that do aggregations (e.g. "Test Runs Summary" gadget). Gadgets that just "list" entities should not affect performance significatively.
2. Image Added Limit the target issues for the reports/gadgets, i.e. generate the Overall Requirement Coverage (report/gadget) just for issues that you really need and not all Jira requirements or projects; the setting "Max number of requirements per report or gadget results" (available under Miscellaneous) acts as a maximum limit for some reports/gadgets.

Dashboards

1. Image Added Choose properly the filters you use for each gadget, in order to restrict the amount of issues that will be processed
2. Image Added Don't use low (i.e. intensive) refresh times as they will add some overhead to the Jira instance
3. Image Added Use shareable dashboards to have the high-level overview and the things that matter but avoid creating highly complex dashboards
4. Image Added Try to normalize the dashboards and make them like a standard organization wide; it will facilitate communication and avoid “wrong”/unoptimized usage

Custom Reports

1. Image Added If possible, use eazyBI for making custom table/chart based reports with drill-down capabilities; it is highly flexible and works in its own database, thus not impacting Jira unless the database is hosted in the same instance of Jira server/database
2. Image Added If using the Xporter app to build custom PDF, Word or Excel based reports, then please see Xporter specific recommendations.

Administration and Customization

Settings

Xray comes with default settings that are ok for most organizations. Nevertheless, have a look at the different settings; it’s an opportunity to know the product better and for you to think on ways to fine tune it later.

1. Image Added Review the settings "Max Test Runs for bulk operations", "Max number of requirements per report or gadget results", "Max number of Tests per Test Plan", "Max results per request" in the the Miscellaneous administration settings. The default values should be ok.
   
2. Image Added Using the requirement coverage strategy "Use versioned Test Sets for Requirement Coverage" will require additional computational resources on daily usage; therefore, the default "Use versioned Test Executions for Requirement Coverage" strategy is the recommend one. Learn more in Requirements Coverage. If you change this setting, then it can take a while until all statuses are re-calculated depending on your instance dimension.

Your own custom fields

1. Image Added Beware with calculated custom fields implemented using some customization app (e.g. Jira Misc Custom Fields, ScriptRunner, etc) as they will be calculated each time they're accessed and they can impact things such as:
   • indexing time
   • REST API calls (e.g. whenever searching issues)
   • on listings (e.g. on Issues search page, Filters Results gadget, Xray tabular sections) if they're included as columns .
2. Image Added Adding a lot of custom fields on Jira will add some overhead on performance; therefore, Atlassian itself recommends a responsible usage of custom fields.

Workflows on Xray entities

1. Image Added Don't use complex workflows for Xray entities as it will harden their usage; if the workflows have many steps then it will impact overall Jira performance

Reindex

1. Image Added Xray provides some maintenance operations that can be used under certain circumstances, if needed. If you clear some of Xray calculated custom fields, either using Integrity Checker > Calculated Custom Field Values option in administration settings or using the available bulk operations for Requirement Status and TestRunStatus custom fields, it will impact the reindex time considerable as the values have to be all recalculated. 
2. Diamantino Campos haverá algum custom field que possa ser excluido do reindex e que afecte o tempo de reindex dramaticamente?

...

Use

...

testPlanTests() - whenever by tests in a given status; the current workaround is to search using the "TestRunStatus" custom field. 

...

When searching for Tests with a certain status inside a Test Plan, we recommend you to use the custom field search instead.

...

Xray calculated custom fields

1. Image Removed  Xray provides some specific custom fields that calculate their values on the fly. This means that you should have that in mind, specially if you're including them in tables/issue listings/gadgets.
   1. The most intensive custom fields are the "Test Execution Defects" and the "Test Set Status". The "Tests Count", as it does an aggregation, also is intensive if you use it for multiple issues.

Reports and Gadgets

1. Image Removed One way of doing reporting is by using gadgets. Gadgets are great to share information between team members and even between different teams; however, if not use carefully, they can degrade JIRA performance if all users have the same report on their dashboard as they will probably generate multiple requests once users access the dashboard. Thus, use carefully most intensive gadgets such as the "Historical Daily Requirement Coverage" and the "Tests Evolution" gadget and others that do aggregations (e.g. "Test Runs Summary" gadget). Gadgets that just "list" entities should not affect performance significatively.
2. Image Removed Limit the target issues for the reports/gadgets, i.e. generate the Overall Requirement Coverage (report/gadget) just for issues that you really need and not all Jira requirements or projects; the setting "Max number of requirements per report or gadget results" (available under Miscellaneous) acts as a maximum limit for some reports/gadgets.

Dashboards

1. Image Removed Choose properly the filters you use for each gadget, in order to restrict the amount of issues that will be processed
2. Image Removed Don't use low (i.e. intensive) refresh times as they will add some overhead to the Jira instance
3. Image Removed Use shareable dashboards to have the high-level overview and the things that matter but avoid creating highly complex dashboards
4. Image Removed Try to normalize the dashboards and make them like a standard organization wide; it will facilitate communication and avoid “wrong”/unoptimized usage

Custom Reports

1. Image Removed If possible, use eazyBI for making custom table/chart based reports with drill-down capabilities; it is highly flexible and works in its own database, thus not impacting Jira unless the database is hosted in the same instance of Jira server/database
2. Image Removed If using the Xporter app to build custom PDF, Word or Excel based reports, then please see Xporter specific recommendations.

Administration and Customization

Settings

Xray comes with default settings that are ok for most organizations. Nevertheless, have a look at the different settings; it’s an opportunity to know the product better and for you to think on ways to fine tune it later.

1. Image Removed Review the settings "Max Test Runs for bulk operations", "Max number of requirements per report or gadget results", "Max number of Tests per Test Plan", "Max results per request" in the the Miscellaneous administration settings. The default values should be ok.
   
2. Image Removed Using the requirement coverage strategy "Use versioned Test Sets for Requirement Coverage" will require additional computational resources on daily usage; therefore, the default "Use versioned Test Executions for Requirement Coverage" strategy is the recommend one. Learn more in Requirements Coverage. If you change this setting, then it can take a while until all statuses are re-calculated depending on your instance dimension.

Your own custom fields

1. Image Removed Beware with calculated custom fields implemented using some customization app (e.g. Jira Misc Custom Fields, ScriptRunner, etc) as they will be calculated each time they're accessed and they can impact things such as:
   • indexing time
   • REST API calls (e.g. whenever searching issues)
   • on listings (e.g. on Issues search page, Filters Results gadget, Xray tabular sections) if they're included as columns .
2. Image Removed Adding a lot of custom fields on Jira will add some overhead on performance; therefore, Atlassian itself recommends a responsible usage of custom fields.

Workflows on Xray entities

1. Image Removed Don't use complex workflows for Xray entities as it will harden their usage; if the workflows have many steps then it will impact overall Jira performance

Reindex???

...

Other operations

1.  Avoid bulk cloning Test plans or Test Executions using some specific apps for that purpose; cloning multiple issues at the same time can add some overhead to Jira as these can affect the calculations of the statuses of Tests and requirements.

...