Anti-Pattern Description

This anti-pattern is about creating/having manual, script based test cases, without test steps.

Manual tests should have clearly defined steps. If a user is not creating steps, it may indicate that they are using the wrong test type and will lead to other testers misinterpretation of what needs to be done. Allowing this anti-pattern to spread will lead to significant issues such as:

• Inconsistent Test Executions;
• Difficulty in reproducing Defects, and ultimately;
• Reduced Test Coverage, since critical actions might have been missed or executed incorrectly. 

Solution

The solution for this anti-pattern goes with internal coaching, clearly defined parameters & standards for test case documentation, recurrent audit, and also automating repetitive tasks where detailed steps are critical. Manual, script/structured test cases, should have a number of steps that explain the intent and the expected goals; we should also avoid adding too much detail into it, as it creates unnecessary bureaucracy and makes our test specification more dependent on the implementation and thus more fragile.

However it may also be the case we are using a wrong Test type, in that case we suggest revisiting the following articles for Generic/Unstructured Test Type.

• Generic/Unstructured Tests in Xray DC
• Generic/Unstructured Tests in Xray Cloud

Audit and Review

We suggest a recurrent audit; you can define its cadence, especially in team meetings or dailies to assist and provide feedback on the spot.

Should you be using Xray Data Center, feel free to use the below JQL as a Saved filter with a Filter Result kind of gadget in a dashboard:

issue in manualTestsWithoutSteps()

or

issue in manualTestsWithoutSteps("saved_filter")

Should look like this:

Filter Results Gadget using the following Saved Filter: "issue in manualTestsWithoutSteps() AND Project = "Your Project Name""

Should you use Xray Cloud, it is on the verge of receiving a sort of insights tool which will give you notice of this anti-pattern and others in a Xray page.

Nonetheless, for the moment, it is possible to get this information using Document Generator. 

Snippet code:

&{for issues} 
#{if (%{'${IssueTypeName}'.equals('Test')})} 
#{if (%{'${TestStepsCount}'.equals('0')})} 
${Key} - ${Summary}
#{end}
#{end}
&{end}

Learn More

One key feature you can make use of is Modular Tests in which previously defined manual tests with respective steps can assist you to easily compose a Test scenario.

• Modular Tests in Xray DC
• Modular Tests in Xray Cloud

Our Academy offers Xray Essentials for Cloud & Xray Essentials for Data Center Course to help you get familiarized with how to use the Modular Test functionality.

In fact we encourage your team to go through all the courses → Xray Academy.