Page History

Versions Compared

Key

This line was added.
This line was removed.
Formatting was changed.

Info

title	What you'll learn

Define tests using CypressSpring Boot
Run the test and push the test report to Xray
Validate in Jira that the test results are available

Note

icon	false
title	Source-code for this tutorial
type	Info

code is available in GitHub

Overview

Spring Framework is a well-known Java framework to build Java based applications, supporting IoC (Inversion of Control) principle.

Sprint Boot provides an oppiniated extension on top of Spring that aims to minimize configuration burden and ease the implementation of applications.

With Spring it's possible to create web applications, REST services, and more.

Prerequisites

Expand
For this example we will use Cypress to write tests that aim to validate the Cypress todo example.the built-in testing facilities provided by Spring to test the application developed in Spring Boot We will need: Access to a Cypress todo example site that we aim to test Cypress Java and Maven installed in your machine

Expand

For this example we will use Cypress to write tests that aim to validate the Cypress todo example.the built-in testing facilities provided by Spring to test the application developed in Spring Boot

We will need:

Access to a Cypress todo example site that we aim to test
Cypress Java and Maven installed in your machine

To start using the Cypress please Spring Boot please follow the Get Started documentation.

The tests consists in validating the operations over todo's elements of the Cypress todo example, for that we have defined several tests to:

Validate that we can add new todo items;
Validate that we can check an item as completed;
Validate that we can filter for completed/uncompleted tasks;
Validate that we can delete all completed tasks.

The target web application is a simple "todos" made available by Cypress.

@DataJpaTest
- a test slice focused on the data layer
- By default, tests annotated with @DataJpaTest are transactional and roll back at the end of each test. They also use an embedded in-memory database (H2).
@WebMvcTest
- a mock MVC testing slice without the rest of the application, focused only on Spring MVC components
@SpringBootTest
- loads the full application
- can use a specific database for testing purposes using @AutoConfigureTestDatabase and @TestPropertySource
@TestEntityManager
- a test-friendly EntityManager that provides additional methods useful for testing

Our Spring application provides:

3 controllers:
- IndexController: that is used to return the text "Welcome to this amazing website!" whenever accessing the root page /
- GreetingController: that is used to return a greeting message (e.g., "Hello, xxx!") based on an HTML template
- UserRestController: that provides a REST API to manage users using several endpoints under the /users base URL
1 service:
- UserService/UserServiceImpl: to perform business logic on users; in this case just as a small layer on top of the repository (UserRepository)
1 entity and 1 associated repository:
- User: a persistable entity
- UserRepository: a JPA repository of User objects

We'll implement tests at several levels:

Quick Start Guide documentation; you can also use Spring initializr to make a working skeleton of a project using Spring and its dependencies.

Usually, Spring applications have these layers:

web/presentation layer
1. controllers, exception handlers, filters, ...
service layer
1. services with business logic
persistence/data layer
1. JPA Repository, Entity
2. database

The target SUT is a web application implemented using Spring Boot, having a REST API to manage users and some controllers that return text acting like typical servlets.

Our Spring application provides:

3 controllers:
- IndexController: that is used to return the text "Welcome to this amazing website!" whenever accessing the root page /
- GreetingController: that is used to return a greeting message (e.g., "Hello, xxx!") based on an HTML template
- UserRestController: that provides a REST API to manage users using several endpoints under the /users base URL
1 service:
- UserService/UserServiceImpl: to perform business logic on users; in this case just as a small layer on top of the repository (UserRepository)
1 entity and 1 associated repository:
- User: a persistable entity
- UserRepository: a JPA repository of User objects

@WebMvcTest
- a mock MVC testing slice without the rest of the application, focused only on Spring MVC components
@SpringBootTest
- loads the full application
- can use a specific database for testing purposes using @AutoConfigureTestDatabase and @TestPropertySource
@TestEntityManager
- a test-friendly EntityManager that provides additional methods useful for testing

We'll implement JUnit 5 tests for all these layers:

web:
- we'll test the IndexController and GreetingController controllers
- we'll test the REST API provided by the UserRestController controller
service:
- we will
service:
- we will unit test the UserServiceImpl, avoiding usage of database, to test the logic of the service; we'll use Mockito to mock responses of the UserRepository
data:
- we'll test the UserRepository in isolation, without loading the web environment
- an in-memory database (H2) will be usedTestEntityManager

Info

test slices:

Spring Boot supports test slicing; the idea is to provides slices of the whole ApplicationContext by loading lesser components and thus provide efficiency

. We'll see more about @DataJpaTest and @WebMvcTest ahead.

To test at the data layer, we can use @DataJpaTest as a test slice to test our UserRepository repository and the User entity.

By default, tests annotated with @DataJpaTest are transactional and roll back at the end of each test. They also use an embedded in-memory database (H2).

To test at web layer we can follow different approaches by annotating the related test classes:

@SpringBootTest
- loads the full application; more resource intensive
- web server port is injected using @LocalServerPort; a friendly REST client can be used by an injected TestRestTemplate
@SpringBootTest(webEnvironment = WebEnvironment.MOCK, classes = ...) + @AutoConfigureMockMvc
- loads the full application except the web server itself
  - Another useful approach is to not start the server at all but to test only the layer below that, where Spring handles the incoming HTTP request and hands it off to your controller. That way, almost all of the full stack is used, and your code will be called in exactly the same way as if it were processing a real HTTP request but without the cost of starting the server
- access to MVC framework is made using an injected reference to MockMvc using @Autowired
@WebMvcTest(xxx.class)
- loads a test slice focused just on the web layer, providing a simplified web environment
- access to MVC framework is made using an injected reference to MockMvc using @Autowired
- usually @WebMvcTest is used in combination with @MockBean or @Import to create any collaborators required by your @Controller beans.

Let's see some examples.

The following code snippet shows usage of @WebMvcTest to test a slice containing just the web layer. In this case we're testing the output of the root page.

Code Block

language	java
title	IndexControllerMockedIT.java
collapse	true

package com.idera.xray.tutorials.springboot;

import static org.hamcrest.Matchers.equalTo;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import com.idera.xray.tutorials.springboot.boundary.IndexController;

// @SpringBootTest
// @AutoConfigureMockMvc; it is implied whenever @WebMvcTest is used

// @WebMvcTest annotation is used to test only the web layer of the application
// It disables full auto-configuration and instead apply only configuration relevant to MVC tests
@WebMvcTest(IndexController.class)
public class IndexControllerMockedIT {

	@Autowired
	private MockMvc mvc;

	@Test
	public void getWelcomeMessage() throws Exception {
		mvc.perform(MockMvcRequestBuilders.get("/").accept(MediaType.TEXT_PLAIN))
				.andExpect(status().isOk())
				.andExpect(content().string(equalTo("Welcome to this amazing website!")));
	}
}

The following code snippet loads the whole application using @SpringBootTest to test the REST API endpoints used to manage users.

Code Block

language	java
title	UserRestControllerIT.java
collapse	true

package com.idera.xray.tutorials.springboot;

import org.json.JSONException;
import org.json.JSONObject;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.util.UriComponentsBuilder;
import org.springframework.boot.test.web.server.LocalServerPort;
import com.idera.xray.tutorials.springboot.data.User;
import com.idera.xray.tutorials.springboot.data.UserRepository;
import app.getxray.xray.junit.customjunitxml.annotations.XrayTest;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;

/* @SpringBootTest loads the full application, including the web server
 * @AutoConfigureTestDatabase is used to configure a test database instead of the application-defined database
*/
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@AutoConfigureTestDatabase
class UserRestControllerIT {

    @LocalServerPort
    int randomServerPort;

    @Autowired
    private TestRestTemplate restTemplate;

    @Autowired
    private UserRepository repository;

    User user1;

    @BeforeEach
    public void resetDb() {
        repository.deleteAll();
        user1 = repository.save(new User("Sergio Freire", "sergiofreire", "dummypassword"));
    }


    @XrayTest(key = "XRAY-1")
    @Test
     void createUserWithSuccess() {
        User john = new User("John Doe", "johndoe", "dummypassword");
        ResponseEntity<User> entity = restTemplate.postForEntity("/api/users", john, User.class);

        List<User> foundUsers = repository.findAll();
        assertThat(foundUsers).extracting(User::getUsername).contains("johndoe");
    }


    @XrayTest(key = "XRAY-2")
    @Test
     void dontCreateUserForInvalidData() {
        User john = new User("John Doe", "", "dummypassword");
        ResponseEntity<User> response = restTemplate.postForEntity("/api/users", john, User.class);
 
        // ideally, the server shouldnt return 500, but 400 (bad request)
        assertThat(response.getStatusCode()).isEqualTo(HttpStatus.INTERNAL_SERVER_ERROR);
 
        List<User> found = repository.findAll();
        assertThat(found).hasSize(1);
        assertThat(found).extracting(User::getName).doesNotContain("John Doe");
    }

    @Test
    void getUserWithSuccess() {
        String endpoint = UriComponentsBuilder.newInstance()
                .scheme("http")
                .host("127.0.0.1")
                .port(randomServerPort)
                .pathSegment("api", "users", user1.getId().toString() )
                .build()
                .toUriString();

        ResponseEntity<User> response = restTemplate.exchange(endpoint, HttpMethod.GET, null, new ParameterizedTypeReference<User>() {
        });
        User user = response.getBody();

        assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
        assertThat(user1.equals(user)).isTrue();
    }

    @Test
    void getUserUnsuccess() throws JSONException {
        /*
        String endpoint = UriComponentsBuilder.newInstance()
                .scheme("http")
                .host("127.0.0.1")
                .port(randomServerPort)
                .pathSegment("api", "users", "-1" )
                .build()
                .toUriString();

        */

        ResponseEntity<JSONObject> response = restTemplate.exchange("/api/user/-1", HttpMethod.GET, null, new ParameterizedTypeReference<JSONObject>() {
        });

        assertThat(response.getStatusCode()).isEqualTo(HttpStatus.NOT_FOUND);
    }
    

    @Test
     void listAllUsersWithSuccess()  {
        createTempUser("Amanda James", "amanda", "dummypassword");
        createTempUser("Robert Junior", "robert", "dummypassword");

        ResponseEntity<List<User>> response = restTemplate
                .exchange("/api/users", HttpMethod.GET, null, new ParameterizedTypeReference<List<User>>() {
                });

        assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
        assertThat(response.getBody()).extracting(User::getName).containsExactly("Sergio Freire", "Amanda James", "Robert Junior");
    }

    @Test
    void deleteUserWithSuccess() {
        ResponseEntity<User> response = restTemplate.exchange("/api/users/" + user1.getId(), HttpMethod.DELETE, null, User.class);
        assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
        assertThat(response.getBody().getName()).isEqualTo("Sergio Freire");

        List<User> found = repository.findAll();
        assertThat(found).isEmpty();
    }

    @Test
    void deleteUserUnsuccess() {
        ResponseEntity<User> response = restTemplate.exchange("/api/users/" + (user1.getId()+2), HttpMethod.DELETE, null, User.class);
        assertThat(response.getStatusCode()).isEqualTo(HttpStatus.NOT_FOUND);

        List<User> found = repository.findAll();
        assertThat(found).hasSize(1);
    }

    private void createTempUser(String name, String username, String password) {
        User user = new User(name, username, password);
        repository.saveAndFlush(user);
    }

}

In our case, we have tests that will be picked by surefire plugin and other ones that will be picked by failsafe plugin.

Once the code is implemented it can be executed with the following command:

Code Block

language	bash
theme	DJango
firstline	1

mvn test failsafe:integration-test

The results are immediately available in the terminal.

Image Added

Ultimately this will lead to multiple JUnit XML reports (in target/surefire-reports/ and target/failsafe-reports/, respectively).

If we use the xray-junit-extensions maven plugin, it will generate 1 JUnit XML report (i.e., in reports/TEST-junit-jupiter.xml) with all results of the last goal (i.e., the integration tests ran by failsafe on the previous mvn command).

In this example, all tests have succeed, as seen in the previous terminal screenshot. It generates the following JUnit XML report.

Code Block

firstline	1
title	JUnit XML Report
collapse	true

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="JUnit Jupiter" tests="11" skipped="0" failures="0" errors="0" time="6" hostname="Sergios-MBP.lan" timestamp="2024-02-12T17:42:55">
<properties>
<property name="CONSOLE_LOG_CHARSET" value="UTF-8"/>
<property name="FILE_LOG_CHARSET" value="UTF-8"/>
<property name="PID" value="45881"/>
<property name="apple.awt.application.name" value="ForkedBooter"/>
<property name="basedir" value="/Users/sergio/exps/tutorial-spring"/>
<property name="catalina.base" value="/private/var/folders/2w/5zhst_816kl0dcfv4t7g3vv00000gn/T/tomcat.0.9109224442063392217"/>
<property name="catalina.home" value="/private/var/folders/2w/5zhst_816kl0dcfv4t7g3vv00000gn/T/tomcat.0.16552584680725469127"/>
<property name="catalina.useNaming" value="false"/>
<property name="com.zaxxer.hikari.pool_number" value="1"/>
<property name="file.encoding" value="UTF-8"/>
<property name="file.separator" value="/"/>
<property name="ftp.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
<property name="http.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
<property name="java.awt.headless" value="true"/>
<property name="java.class.version" value="65.0"/>
<property name="java.home" value="/usr/local/Cellar/openjdk/21.0.1/libexec/openjdk.jdk/Contents/Home"/>
<property name="java.io.tmpdir" value="/var/folders/2w/5zhst_816kl0dcfv4t7g3vv00000gn/T/"/>
<property name="java.runtime.name" value="OpenJDK Runtime Environment"/>
<property name="java.runtime.version" value="21.0.1"/>
<property name="java.specification.name" value="Java Platform API Specification"/>
<property name="java.specification.vendor" value="Oracle Corporation"/>
<property name="java.specification.version" value="21"/>
<property name="java.vendor" value="Homebrew"/>
<property name="java.vendor.url" value="https://github.com/Homebrew/homebrew-core/issues"/>
<property name="java.vendor.url.bug" value="https://github.com/Homebrew/homebrew-core/issues"/>
<property name="java.vendor.version" value="Homebrew"/>
<property name="java.version" value="21.0.1"/>
<property name="java.version.date" value="2023-10-17"/>
<property name="java.vm.compressedOopsMode" value="Zero based"/>
<property name="java.vm.info" value="mixed mode, sharing"/>
<property name="java.vm.name" value="OpenJDK 64-Bit Server VM"/>
<property name="java.vm.specification.name" value="Java Virtual Machine Specification"/>
<property name="java.vm.specification.vendor" value="Oracle Corporation"/>
<property name="java.vm.specification.version" value="21"/>
<property name="java.vm.vendor" value="Homebrew"/>
<property name="java.vm.version" value="21.0.1"/>
<property name="jdk.debug" value="release"/>
<property name="line.separator" value="
"/>
<property name="localRepository" value="/Users/sergio/.m2/repository"/>
<property name="native.encoding" value="UTF-8"/>
<property name="org.jboss.logging.provider" value="slf4j"/>
<property name="os.arch" value="x86_64"/>
<property name="os.name" value="Mac OS X"/>
<property name="os.version" value="14.2.1"/>
<property name="path.separator" value=":"/>
<property name="socksNonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
<property name="stderr.encoding" value="UTF-8"/>
<property name="stdout.encoding" value="UTF-8"/>
<property name="sun.arch.data.model" value="64"/>
<property name="sun.boot.library.path" value="/usr/local/Cellar/openjdk/21.0.1/libexec/openjdk.jdk/Contents/Home/lib"/>
<property name="sun.cpu.endian" value="little"/>
<property name="sun.io.unicode.encoding" value="UnicodeBig"/>
<property name="sun.java.launcher" value="SUN_STANDARD"/>
<property name="sun.jnu.encoding" value="UTF-8"/>
<property name="sun.management.compiler" value="HotSpot 64-Bit Tiered Compilers"/>
<property name="user.country" value="PT"/>
<property name="user.dir" value="/Users/sergio/exps/tutorial-spring"/>
<property name="user.home" value="/Users/sergio"/>
<property name="user.language" value="en"/>
<property name="user.name" value="sergio"/>
<property name="user.timezone" value="Europe/Lisbon"/>
</properties>
<testcase name="getPersonalizedGreeting" classname="com.idera.xray.tutorials.springboot.GreetingControllerMockedIT" time="0" started-at="2024-02-12T17:42:53.713267" finished-at="2024-02-12T17:42:54.038343">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.GreetingControllerMockedIT]/[method:getPersonalizedGreeting()]
display-name: getPersonalizedGreeting()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="dontCreateUserForInvalidData" classname="com.idera.xray.tutorials.springboot.UserRestControllerIT" time="0" started-at="2024-02-12T17:42:54.540558" finished-at="2024-02-12T17:42:54.82271">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.UserRestControllerIT]/[method:dontCreateUserForInvalidData()]
display-name: dontCreateUserForInvalidData()
]]></system-out>
<properties>
<property name="test_key" value="XRAY-2"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="getUserUnsuccess" classname="com.idera.xray.tutorials.springboot.UserRestControllerIT" time="0" started-at="2024-02-12T17:42:54.896158" finished-at="2024-02-12T17:42:54.917811">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.UserRestControllerIT]/[method:getUserUnsuccess()]
display-name: getUserUnsuccess()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="deleteUserWithSuccess" classname="com.idera.xray.tutorials.springboot.UserRestControllerIT" time="0" started-at="2024-02-12T17:42:54.856928" finished-at="2024-02-12T17:42:54.876068">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.UserRestControllerIT]/[method:deleteUserWithSuccess()]
display-name: deleteUserWithSuccess()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="createUserWithSuccess" classname="com.idera.xray.tutorials.springboot.UserRestControllerIT" time="0" started-at="2024-02-12T17:42:54.876839" finished-at="2024-02-12T17:42:54.895425">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.UserRestControllerIT]/[method:createUserWithSuccess()]
display-name: createUserWithSuccess()
]]></system-out>
<properties>
<property name="test_key" value="XRAY-1"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="getDefaultGreeting" classname="com.idera.xray.tutorials.springboot.GreetingControllerMockedIT" time="0" started-at="2024-02-12T17:42:54.039385" finished-at="2024-02-12T17:42:54.044234">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.GreetingControllerMockedIT]/[method:getDefaultGreeting()]
display-name: getDefaultGreeting()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="getWelcomeMessage" classname="com.idera.xray.tutorials.springboot.IndexControllerMockedIT" time="0" started-at="2024-02-12T17:42:55.133942" finished-at="2024-02-12T17:42:55.143765">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.IndexControllerMockedIT]/[method:getWelcomeMessage()]
display-name: getWelcomeMessage()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="getUserWithSuccess" classname="com.idera.xray.tutorials.springboot.UserRestControllerIT" time="0" started-at="2024-02-12T17:42:54.823735" finished-at="2024-02-12T17:42:54.856158">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.UserRestControllerIT]/[method:getUserWithSuccess()]
display-name: getUserWithSuccess()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="listAllUsersWithSuccess" classname="com.idera.xray.tutorials.springboot.UserRestControllerIT" time="0" started-at="2024-02-12T17:42:54.936475" finished-at="2024-02-12T17:42:54.958408">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.UserRestControllerIT]/[method:listAllUsersWithSuccess()]
display-name: listAllUsersWithSuccess()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="getWelcomeMessage" classname="com.idera.xray.tutorials.springboot.IndexControllerIT" time="0" started-at="2024-02-12T17:42:52.693447" finished-at="2024-02-12T17:42:53.384655">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.IndexControllerIT]/[method:getWelcomeMessage()]
display-name: getWelcomeMessage()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="deleteUserUnsuccess" classname="com.idera.xray.tutorials.springboot.UserRestControllerIT" time="0" started-at="2024-02-12T17:42:54.918649" finished-at="2024-02-12T17:42:54.935546">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:com.idera.xray.tutorials.springboot.UserRestControllerIT]/[method:deleteUserUnsuccess()]
display-name: deleteUserUnsuccess()
]]></system-out>
<properties>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]
display-name: JUnit Jupiter
]]></system-out>
</testsuite>

We can use the @DataJpaTest to fo

https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.testing.spring-boot-applications.autoconfigured-tests

Each of these tests will have a series of actions and validations to check that the desired behavior is happening as we can see below:

Code Block

language	js
title	todo.cy.js
collapse	true

describe('example to-do app', () => {
  beforeEach(() => {
    cy.visit(Cypress.config('baseUrl'))
  })

  it('can add new todo items', () => {
    const newItem = 'Feed the cat'
    cy.get('[data-test=new-todo]').type(`${newItem}{enter}`)

    cy.get('.todo-list li')
      .should('have.length', 3)
      .last()
      .should('have.text', newItem)
  })

  it('can check an item as completed', () => {
    cy.contains('Pay electric bill')
      .parent()
      .find('input[type=checkbox]')
      .check()

    cy.contains('Pay electric bill')
      .parents('li')
      .should('have.class', 'completed')
  })

  context('with a checked task', () => {
    beforeEach(() => {
      cy.contains('Pay electric bill')
        .parent()
        .find('input[type=checkbox]')
        .check()
    })

    it('can filter for uncompleted tasks', () => {
      cy.contains('Active').click()

      cy.get('.todo-list li')
        .should('have.length', 1)
        .first()
        .should('have.text', 'Walk the dog')

      cy.contains('Pay electric bill').should('not.exist')
    })

    it('can filter for completed tasks', () => {
      cy.contains('Completed').click()

      cy.get('.todo-list li')
        .should('have.length', 1)
        .first()
        .should('have.text', 'Pay electric bill')

      cy.contains('Walk the dog').should('not.exist')
    })

    it('can delete all completed tasks', () => {
      cy.contains('Clear completed').click()

      cy.get('.todo-list li')
        .should('have.length', 1)
        .should('not.have.text', 'Pay electric bill')

      cy.contains('Clear completed').should('not.exist')
    })
  })
})

The tests are simple but let's look into two diferences that allow a little more control, the first one is the possibility to use hooks like beforeEach to, as the name implies, execute some operations before each test execution. In this example we are accessing the target page before each test avoiding repeating this instruction in each test.

Code Block

language	js
title	beforeEach
collapse	true

...
beforeEach(() => {
  cy.visit('https://example.cypress.io/todo')
})
...

The other one helps in the test organization and have a direct effect on how the results will be written in the result file, in our case we are using context (but we could use describe or specify). This will group the tests beneath into the same testsuite.

Code Block

language	js
title	context
collapse	true

...
  context('with a checked task', () => {
...

These tests are defined to validate the application ability to manage todo's by accessing the Cypress todo example and performing operations that will generate an expected output.

Once the code is implemented it can be executed with the following command:

Code Block

language	bash
theme	DJango
firstline	1

npx cypress run

The results are immediately available in the terminal.

Image Removed

In this example, all tests have succeed, as seen in the previous terminal screenshot. It generates the following JUnit XML report.

Code Block

firstline	1
title	Junit Report
collapse	true

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="4.404" tests="6" failures="0">
  <testsuite name="Root Suite" timestamp="2023-01-30T17:46:57" tests="0" file="cypress/e2e/todo.cy.js" time="0.000" failures="0">
  </testsuite>
  <testsuite name="example to-do app" timestamp="2023-01-30T17:46:57" tests="3" time="0.000" failures="0">
    <testcase name="example to-do app displays two todo items by default" time="0.842" classname="displays two todo items by default">
    </testcase>
    <testcase name="example to-do app can add new todo items" time="0.477" classname="can add new todo items">
    </testcase>
    <testcase name="example to-do app can check off an item as completed" time="0.267" classname="can check off an item as completed">
    </testcase>
  </testsuite>
  <testsuite name="with a checked task" timestamp="2023-01-30T17:47:00" tests="3" time="1.060" failures="0">
    <testcase name="example to-do app with a checked task can filter for uncompleted tasks" time="0.345" classname="can filter for uncompleted tasks">
    </testcase>
    <testcase name="example to-do app with a checked task can filter for completed tasks" time="0.350" classname="can filter for completed tasks">
    </testcase>
    <testcase name="example to-do app with a checked task can delete all completed tasks" time="0.341" classname="can delete all completed tasks">
    </testcase>
  </testsuite>
</testsuites>

Notes:

You can invoke Cypress locally and use it to assist you to write and execute tests with: npx cypress open
Use cypress.config.js to define configuration values such as taking screenshots, recordings or the reporter to use (more info here).
Different parameters can be used in the command line (more info here)
We are using JUnit reporter but others are available (more info here)

Integrating with Xray

As we saw in the previous example, where we are producing JUnit reports with the test results. It is now a matter of importing those results to your Jira instance; this can be done by simply submitting automation results to Xray through the REST API, by using one of the available CI/CD plugins (e.g. for Jenkins), or using the Jira interface to do so.

UI Tabs

UI Tab

title	API

API

Once you have the report file available you can upload it to Xray through a request to the REST API endpoint for JUnit, and for that the first step is to follow the instructions in v1 or v2 (depending on your usage) to obtain the token we will be using in the subsequent requests.

JUnit XML results

We will use the API request with the definition of some common fields on the Test Execution, such as the target project, project version, etc.

In the first version of the API, the authentication used a login and password (not the token that is used in Cloud).

Code Block

language	bash
theme	DJango

curl -H "Content-Type: multipart/form-data" -u admin:admin -F "file=@reports/TEST-junit-jupiter.xml" 'http://<LOCAL_JIRA_INSTANCE>/rest/raven/2.0/import/execution/junit?projectKey=XT&testPlanKey=XT-401436'

With this command we are creating a new Test Execution in the referred Test Plan with a generic summary and six tests with a summary based on the test name.based on the test name. One of the tests was not auto-provisioned as it already existed beforehand and was referred in the the test code using the @XrayTest annotation.

Image AddedImage Removed

UI Tab

title	Jira UI

Jira UI

UI Steps

UI Step

Create a Test Execution linked to the Test Plan that you have.

Image RemovedImage Added

UI Step

Fill in the necessary fields and press "Create".

Image RemovedImage Added

UI Step

Open the Test Execution and import the JUnit report

Image RemovedImage Added

UI Step

Choose the results file and press "Import"

Image Modified

UI Step

The Test Execution is now updated with the test results imported.

Image RemovedImage Added

Tests implemented using Cypress Junit will have a corresponding Test entity in Xray. Once results are uploaded, Test issues corresponding to the Cypress tests are auto-provisioned (e.g., XT-440), unless they already exist; in our case, we have explicitly mentioned an existing Test issue (XT-438) from one of the tests.

Image RemovedImage Added Image Added

Xray uses a concatenation of the suite name and the test name as the the unique identifier for the test.

In Xray, results are stored in a Test Execution, usually a new one. The Test Execution contains a Test Run per each test that was executed using Cypress.

Image RemovedImage Added

Detailed results, including logs and exceptions reported during execution of the test, can be seen on the execution screen details of each Test Run, accessible through the Execution details:

Image RemovedImage Added

As we can see here:

Image RemovedImage Added

Tips

after results are imported, in Jira Tests can be linked to existing requirements/user stories, so you can track the impacts on their coverage.
results from multiple builds can be linked to an existing Test Plan, to facilitate the analysis of test result trends across builds.
results can be associated with a Test Environment, in case you want to analyze coverage and test results by that environment later on. A Test Environment can be a testing stage (e.g. dev, staging, preprod, prod) or a identifier of the device/application used to interact with the system (e.g. browser, mobile OS).

References

Table of Contents

class	toc-btf

CSS Stylesheet
.toc-btf { position: fixed; }

Content

Space Tools

Versions Compared

Old Version 2

New Version 3

Key

Overview

Prerequisites

Integrating with Xray

API

JUnit XML results

Jira UI

Tips

References