Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1.  Don't add many custom fields to Tests as it will add some additional overhead to Jira.

  2. Image Modified Promote reusability and avoid cloning Test cases, if they’re the same. A Test can be reused multiple times and can be used to cover more than one requirement (if really needed), no matter in which project it is located.Image Removed We also recommend up to 5 linked Tests per each requirement; ideally a Test should be focused on the validation of one requirement.
    • This recommendation helps improve performance both on the TestRun Status TestRunStatus and Requirement Status calculations and also when loading the issue screens and reports.

  3. Image Added Promote reusability and avoid cloning Test cases, if they’re the same. A Test can be reused multiple times and can be used to cover more than one requirement (if really needed), no matter in which project it is located.

Pre-Conditions

  1.  Although the impact is neglectable, try to use Pre-Conditions as means to have manageable end reusable initial conditions that you can link to multiple Tests. That will avoid creating additional steps in all those test cases.

...

  1.  Reuse existing Test Executions (and related Test Runs), if possible, to make calculations faster througout the application and thus make reports and some panels, for example, also faster
    1. if you don't need historical data of interim builds, you may reuse an existing Test Execution and update the corresponding Test Runs. During CI, depending on the endpoint being used, you may specify a Test Execution issue key. In such case, Xray will overwrite the existing Test Runs information, or add new ones if the Test is not yet part of the Test Execution.
  2. Image Added Clean-up old, unneeded executions related data, to make calculations faster througout the application and thus make reports and some panels, for example, also fasterImage Removed Cleanup
    1. If your organization performs a high number of Test Executions (consequently creating also a high number of Test Runs) we recommend deleting old Test Executions issues from time to time (e. g.1 year). This recommendation is mostly for applies to organizations that import lots of many automated executions daily using the Xray REST API.

      The clean-up process must be scheduled for a period when there is a low Jira usage period because when Test Executions are deleted, Xray will re-calculate TestRun Status TestRunStatus and Requirement Status. This might temporarily slow down the Jira instance depending on the amount of issues affected.

    2. Deleting Test Runs can affect the calculated and consolidated status of your Tests and of your requirements for all scopes (e.g. versions or Test Plans + Test Environments); please use carefully
  3.  Although there isn’t an hard limit, we recommend having no more than 2000 Tests in a given Test Execution mostly to ease their management. This limit may be easily superseded depending on Jira instance deployment configuration.

...

Automation & Continuous Integration

  1. Upload Image Added Upload only relevant test results (e.g. don’t upload unit test results); choose properly what testing results you want to track within Xray
  2. Choose Image Added Choose properly the upload frequency
    1. aggregate relevant results in some job run periodically (e.g. hourly, daily) and submit those
  3. Don’t Image Added Don’t mix the CI tool with TM tool
    • Leave the highly detailed execution info on the CI tool

...

  1.  don't create dozens or hundreds of Test Environments; don't try to do data-driven testing using test environments
    • it will impact the calculations that need to be done and the Lucene index

JQL

  1. Some Image Added Some JQL functions, such as the ones dealing with requirement coverage, may be more intensive than other ones, since Xray may have to, for example, load all the related Test Runs in order to obtain relevant data. The following JQL 
    1. requirements(), parentRequirements() , testPlanTests(),
    2. testExecutionTests() - whenever filtering by tests/requirements in a given status; it will depend on the amount of Tests you have on the Test Execution
    3. parentRequirements()  - depending on the amount of requirements and sub-requirements you have, it can take a while to complete and require some resources

Diamantino Campos , Any other Jql function?

REST API

  1. Image Modified Review the "Max results per request" setting in the the Miscellaneous administration settings as it controls the pagination on the REST API calls. The default value should be ok.
  2. Image Modified export results 
  3. https://confluence.xpand-it.com/display/XRAY/Export+Execution+Results+-+REST
  4. Image Added Limit API calls (Jira and Xray related ones) using a reverse proxy
    1. Evaluate what REST API calls are being used, discuss their real need with users
      1. Make sure that pagination is being used on the REST API calls
    2. Restrict access to REST API calls
      1. Limit access to well-known hosts/applications

Xray calculated custom fields

  1.  Xray provides some specific custom fields that calculate their values on the fly. This means that you should have that in mind, specially if you're including them in tables/issue listings/gadgets.

...

  1.  Beware with calculated custom fields implemented using some customization app (e.g. Jira Misc Custom Fields, ScriptRunner, etc) as they will be calculated each time they're accessed and they can impact things such as
    • indexing time
    • REST API calls (e.g. whenever searching issues)
    • on listings (e.g. on Issues search page, Filters Results gadget, Xray tabular sections) if they're included as columns 
  2.  Adding a lot of custom fields on JIira on Jira will add some overhead on performance; therefore, Atlassian itself recommends a responsible usage of custom fields

...

  1.  One way of doing reporting is by using gadgets. Gadgets are great to share information between team members and even between different teams; however, if not use carefully, they can degrade JIRA performance if all users have the same report on their dashboard as they will probably generate multiple requests once users access the dashboard. Thus, use carefully most intensive gadgets such as the "Historical Daily Requirement Coverage" and the "Tests Evolution" gadget and others that do aggregations (e.g. "Test Runs Summary" gadget). Gadgets that just "list" entities should not affect performance significatively.
  2.  Limit the target issues for the reports/gadget, e.g. generate the Overall Requirement Coverage (report/gadget) just for issues that you really need and not all Jira requirements or projects.

...

  1.  If possible, use eazyBI for making custom table/chart based reports with drill-down capabilities; it is highly flexible and works in its own database, thus not impacting Jira unless the database is hosted in the same instance of Jira server/database

  2.  If using the Xporter app to build custom PDF, Word or Excel based reports, then please see Xporter specific recommendations.


Data Center

  1. Image Added Review Atlassian's best practices for Data Center and the Node sizing overview for Atlassian Data Center article
  2. Image Added Configure your load balancer properly to use dedicated nodes for REST API calls, which can reduce impact on other application nodes
  3. Integration with other apps
    1. Image Added See eazyBI's Data Center related recommendations here

References