Versions Compared

Old Version 12

changes.mady.by.user Sergio Freire

Saved on

compared with

New Version 13

changes.mady.by.user Sergio Freire

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1.  Instead of creating one Test Plan for your release, you may create multiple Test Plans to track different Tests (e.g. manual vs automated); this may be useful if you want to have clear visibility of how certain groups of Tests are progressing and if their execution lifecycle is different from other ones. It will also make your Test Plans considerable lighter.
  2.  If adopting Scrum, create Test Plans per Sprint, to track the testing being done in the scope of your Sprint; manage them as artifacts of your Sprint and add them to your Scrum boards so everyone sees their progress. Per each Sprint you may have more than one Test Plan; check out some possible usage patterns here. 

Execution

...

  1. Image Added don't create dozens or hundreds of Test Environments; don't try to do data-driven testing using test environments
    • it will impact the calculations that need to be done and the Lucene index

Entities

Tests

  1.  Don't add many custom fields to Tests as it will add some additional overhead to Jira.
  2.  We recommend up to 5 linked Tests per each requirement; ideally a Test should be focused on the validation of one requirement.

    • This recommendation helps improve performance both on the TestRunStatus and Requirement Status calculations and also when loading the issue screens and reports.

  3.  Promote reusability and avoid cloning Test cases, if they’re the same. A Test can be reused multiple times and can be used to cover more than one requirement (if really needed), no matter in which project it is located.

...

  1.  Using the Test Plan Board as means to do operations over certain Tests of the Test Plan (e.g. schedule Test Executions for them) and to track results can be more efficient than using the Test Plan issue screen. Although the previous two are not equivalent, the Board provides essentialy the same operations while being lighter most of the times as it not shows all the information you can see in the Test Plan issue screen.
  2.  Avoid many folders shown at the same time as it will impact browser performance at some time. You can do this by limiting of direct child folders you create at a given parent folder and by using the "Expand all" moderately. The amount of folders you have does not affect by itself your Jira backend performance though.

Integrations

Automation & Continuous Integration

  1.  Upload only relevant test results (e.g. don’t upload unit test results); choose properly what testing results you want to track within Xray
  2.  Choose properly the upload frequency
    1. aggregate relevant results in some job run periodically (e.g. hourly, daily) and submit those
  3.  Don’t mix the CI tool with TM tool
    • Leave the highly detailed execution info on the CI tool

Test Environments

  1. Image Removed don't create dozens or hundreds of Test Environments; don't try to do data-driven testing using test environments
    • it will impact the calculations that need to be done and the Lucene index

JQL

Xray provides dozens of JQL functions but you have to use them carefuly to make sure your instance is not affected. Please do train your users on JQL before "allowing" to use them throughout Jira.

...

REST API

  1. Image Added Review the "Max results per request" setting in the the Miscellaneous administration settings as it controls the pagination on the REST API calls. The default value should be ok.
  2. Image Added export results 
  3. https://confluence.xpand-it.com/display/XRAY/Export+Execution+Results+-+REST
  4. Image Added Limit API calls (Jira and Xray related ones) using a reverse proxy
    1. Evaluate what REST API calls are being used, discuss their real need with users
      1. Make sure that pagination is being used on the REST API calls
    2. Restrict access to REST API calls
      1. Limit access to well-known hosts/applications

Reporting

JQL

Xray provides dozens of JQL functions but you have to use them carefuly to make sure your instance is not affected. Please do train your users on JQL before "allowing" to use them throughout Jira.

  1.  Image Added Using unoptimized JQL queries can degrade performance substantially
    1. most times this happens because users don't understand how JQL works first of all; JQL is not like SQL (see Understanding JQL Performance). Thus, filtering issues by project by adding the "project = <xxx>" clause is not the same as specifying the project as argument to the subsequent JQL function.
      1. Example: 

        1. Use

          issue in requirements('OK','CALC')

          ...instead of ...

          project = 'CALC' and issue in requirements('OK')

  2. Image Added Some JQL functions, such as the ones dealing with requirement coverage, may be more intensive than other ones, since Xray may have to, for example, load all the related Test Runs in order to obtain relevant data. The following JQL 
    1. requirements(), , testPlanTests() - 
    2. testExecutionTests() - whenever filtering by tests/requirements in a given status; it will depend on the amount of Tests you have on the Test Execution
    3. parentRequirements()  - depending on the amount of requirements and sub-requirements you have, it can take a while to complete and require some resources

Diamantino Campos , Any other Jql function?


Xray calculated custom fields

  1. Image Added Xray provides

...

Use

...

  1. requirements(), , testPlanTests() - 
  2. testExecutionTests() - whenever filtering by tests/requirements in a given status; it will depend on the amount of Tests you have on the Test Execution
  3. parentRequirements()  - depending on the amount of requirements and sub-requirements you have, it can take a while to complete and require some resources

Diamantino Campos , Any other Jql function?

REST API

  1. Image Removed Review the "Max results per request" setting in the the Miscellaneous administration settings as it controls the pagination on the REST API calls. The default value should be ok.
  2. Image Removed export results 
  3. https://confluence.xpand-it.com/display/XRAY/Export+Execution+Results+-+REST
  4. Image Removed Limit API calls (Jira and Xray related ones) using a reverse proxy
    1. Evaluate what REST API calls are being used, discuss their real need with users
      1. Make sure that pagination is being used on the REST API calls
    2. Restrict access to REST API calls
      1. Limit access to well-known hosts/applications

Xray calculated custom fields

  1. Image Removed Xray provides some specific custom fields that calculate their values on the fly. This means that you should have that in mind, specially if you're including them in tables/issue listings/gadgets.

Your own custom fields

  1. Image Removed Beware with calculated custom fields implemented using some customization app (e.g. Jira Misc Custom Fields, ScriptRunner, etc) as they will be calculated each time they're accessed and they can impact things such as
    • indexing time
    • REST API calls (e.g. whenever searching issues)
    • on listings (e.g. on Issues search page, Filters Results gadget, Xray tabular sections) if they're included as columns 
  2. Image Removed Adding a lot of custom fields on Jira will add some overhead on performance; therefore, Atlassian itself recommends a responsible usage of custom fields

Workflows on Xray entities

  1. Image Removed Don't use complex workflows for Xray entities as it will harden their usage

Settings

Xray comes with default settings that are ok for most organizations. Nevertheless, have a look at the different settings; it’s an opportunity to know the product better and for you to think on ways to fine tune it later.

...


Reports and Gadgets

  1. Image Added One way of doing reporting is by using gadgets. Gadgets are great to share information between team members and even between different teams; however, if not use carefully, they can degrade JIRA performance if all users have the same report on their dashboard as they will probably generate multiple requests once users access the dashboard. Thus, use carefully most intensive gadgets such as the "Historical Daily Requirement Coverage" and the "Tests Evolution" gadget and others that do aggregations (e.g. "Test Runs Summary" gadget). Gadgets that just "list" entities should not affect performance significatively.
  2. Image Added Limit the target issues for the reports/gadget, e.g. generate the Overall Requirement Coverage (report/gadget) just for issues that you really need and not all Jira requirements or projects.

Dashboards

  1. Use dashboards to have the high-level overview and the things that matter
  2. Avoid creating highly complex dashboards or dashboards
  3. Try to normalize the dashboards and make them like a standard organization wid
    1. facilitates communication
    2. avoids “wrong” usage
  4. Choose properly the filters you use for each gadget, in order to restrict the amount of issues that will be processed

Custom Reports

  1. Image Added If possible, use eazyBI for making custom table/chart based reports with drill-down capabilities; it is highly flexible and works in its own database, thus not impacting Jira unless the database is hosted in the same instance of Jira server/database
  2. Image Added If using the Xporter app to build custom PDF, Word or Excel based reports, then please see Xporter specific recommendations.

Administration and Customization

Settings

Xray comes with default settings that are ok for most organizations. Nevertheless, have a look at the different settings; it’s an opportunity to know the product better and for you to think on ways to fine tune it later.

  1. Image Added Review the settings "Max Test Runs for bulk operations", "Max number of requirements per report or gadget results", "Max number of Tests per Test Plan", "Max results per request" in the the Miscellaneous administration settings. The default values should be ok.

  • quais outros settings podem ter mais impacto na performance? separation of concerns?

Your own custom fields

  1. Image Added Beware with calculated custom fields implemented using some customization app (e.g. Jira Misc Custom Fields, ScriptRunner, etc) as they will be calculated each time they're accessed and they can impact things such as
    • indexing time
    • REST API calls (e.g. whenever searching issues)
    • on listings (e.g. on Issues search page, Filters Results gadget, Xray tabular sections) if they're included as columns 
  2. Image Added Adding a lot of custom fields on Jira will add some overhead on performance; therefore, Atlassian itself recommends a responsible usage of custom fields

Workflows on Xray entities

  1. Image Added Don't use complex workflows for Xray entities as it will harden their usage
  • quais outros settings podem ter mais impacto na performance? separation of concerns?

Reports and Gadgets

  1. Image Removed One way of doing reporting is by using gadgets. Gadgets are great to share information between team members and even between different teams; however, if not use carefully, they can degrade JIRA performance if all users have the same report on their dashboard as they will probably generate multiple requests once users access the dashboard. Thus, use carefully most intensive gadgets such as the "Historical Daily Requirement Coverage" and the "Tests Evolution" gadget and others that do aggregations (e.g. "Test Runs Summary" gadget). Gadgets that just "list" entities should not affect performance significatively.
  2. Image Removed Limit the target issues for the reports/gadget, e.g. generate the Overall Requirement Coverage (report/gadget) just for issues that you really need and not all Jira requirements or projects.

Dashboards

  1. Use dashboards to have the high-level overview and the things that matter
  2. Avoid creating highly complex dashboards or dashboards
  3. Try to normalize the dashboards and make them like a standard organization wid
    1. facilitates communication
    2. avoids “wrong” usage
  4. Choose properly the filters you use for each gadget, in order to restrict the amount of issues that will be processed

Custom Reports

  1. Image Removed If possible, use eazyBI for making custom table/chart based reports with drill-down capabilities; it is highly flexible and works in its own database, thus not impacting Jira unless the database is hosted in the same instance of Jira server/database
  2. Image Removed If using the Xporter app to build custom PDF, Word or Excel based reports, then please see Xporter specific recommendations.


Data Center Tips

  1.  Review Atlassian's best practices for Data Center and the Node sizing overview for Atlassian Data Center article
  2.  Configure your load balancer properly to use dedicated nodes for REST API calls, which can reduce impact on other application nodes
  3. Integration with other apps
    1.  See eazyBI's Data Center related recommendations here

...