Overview

In this tutorial, we will execute some tests using the Robot Framework. This tutorial explores the specific integration Xray provides for Robot Framework XML reports.

Requirements

  • Robot Framework
  • Selenium2Library
  • Java (if using the Java variant of the "robot framework")

Description


Below are several Robot test suites, each one containing several Robot test cases. 

login_tests/gherkin_login.robot 
*** Settings ***
Documentation A test suite with a single Gherkin style test.
...
... This test is functionally identical to the example in
... valid_login.robot file.
Resource resource.robot
Test Teardown Close Browser
*** Test Cases ***
Gherkin Valid Login
 [Tags] CALC-1 CALC-42
 Given browser is opened to login page
 When user "admin" logs in with password "admin"
 Then welcome page should be open
*** Keywords ***
Browser is opened to login page
 Open browser to login page
User "${username}" logs in with password "${password}"
 Input username ${username}
 Input password ${password}
 Submit credentials
login_tests/invalid_login.robot 
*** Settings ***
Documentation     A test suite containing tests related to invalid login.
...
...               These tests are data-driven by their nature. They use a single
...               keyword, specified with Test Template setting, that is called
...               with different arguments to cover different scenarios.
...
...               This suite also demonstrates using setups and teardowns in
...               different levels.
Suite Setup       Open Browser To Login Page
Suite Teardown    Close Browser
Test Setup        Go To Login Page
Test Template     Login With Invalid Credentials Should Fail
Resource          resource.robot
*** Test Cases ***               USER NAME        PASSWORD
Invalid Username                 invalid          ${VALID PASSWORD}
Invalid Password                 ${VALID USER}    invalid
Invalid Username And Password    invalid          whatever
Empty Username                   ${EMPTY}         ${VALID PASSWORD}
Empty Password                   ${VALID USER}    ${EMPTY}
Empty Username And Password      ${EMPTY}         ${EMPTY}
*** Keywords ***
Login With Invalid Credentials Should Fail
    [Arguments]    ${username}    ${password}
    Input Username    ${username}
    Input Password    ${password}
    Submit Credentials
    Login Should Have Failed
Login Should Have Failed
    Page Should Contain Element     xpath=//*[@id="login-form"]/div[1]/div[@class="aui-message error"]
    ${txt}=                         Get Text            xpath=//*[@id="login-form"]/div[1]/div[@class="aui-message error"]/p
    Should Be Equal As Strings      ${txt}              Sorry, your username and password are incorrect - please try again.
    Title Should Be                 Log in - Xray Demo Environment (JIRA 7)
login_tests/valid_login.robot
*** Settings ***
Documentation     A test suite with a single test for valid login.
...
...               This test has a workflow that is created using keywords in
...               the imported resource file.
Resource          resource.robot
*** Test Cases ***
Valid Login
    [Tags]  CALC-1  CALC-2
    Open Browser To Login Page
    Input Username    admin
    Input Password    admin
    Submit Credentials
    Welcome Page Should Be Open
    [Teardown]    Close Browser

The "CALC-1" and "CALC-2" tags above can be used to link the test case to existing requirement(s) or to an existing Test. When a requirement issue key is given, a link between test and requirement is created.


The previous Robot files use a common resource that contains some generic variables and some reusable "keywords" (i.e., steps).

login_tests/resource.robot 
*** Settings ***
Documentation     A resource file with reusable keywords and variables.
...
...               The system specific keywords created here form our own
...               domain specific language. They utilize keywords provided
...               by the imported Selenium2Library.
Library         BuiltIn
Library         Selenium2Library
*** Variables ***
${SERVER}         localhost:8080
${BROWSER}        chrome
${DELAY}          0.5
${VALID USER}       admin
${VALID PASSWORD}   admin
${LOGIN URL}      http://${SERVER}/login.jsp
${WELCOME URL}    http://${SERVER}/secure/Dashboard.jspa
*** Keywords ***
Open Browser To Login Page
    Open Browser    ${LOGIN URL}    ${BROWSER}
    Maximize Browser Window
    Set Selenium Speed    ${DELAY}
    Login Page Should Be Open
Login Page Should Be Open
    Title Should Be    Log in - Xray Demo Environment (JIRA 7)
Go To Login Page
    Go To    ${LOGIN URL}
    Run Keyword And Ignore Error        Get Alert Message
    Login Page Should Be Open
Input Username
    [Arguments]    ${username}
    Input Text    login-form-username    ${username}
Input Password
    [Arguments]    ${password}
    Input Text    login-form-password    ${password}
Submit Credentials
    Click Button    login-form-submit
Welcome Page Should Be Open
    Location Should Be    ${WELCOME URL}
    Title Should Be    Tests Dashboard - Xray Demo Environment (JIRA 7)


After running the tests and generating the Robot XML report (e.g., output.xml), it can be imported to Xray via the REST API.


If you're using Python:

robot -d output .


If you're using Java:

java -jar robotframework-3.0.jar -d output  .


Each Robot's test case is mapped to a Generic Test in Jira, having the summary with the name of the test case, and the Generic Test Definition field contains the concatenated names of the test suites along with the name of the test case. Note that Robot Framework considers the base folder of the project as the first test suite. The way you run your tests also affects Robot's XML; so, if you execute the file from somewhere else or you execute the file directly by passing it as an argument, the test suite's information will potentially be different.



You will see information about each Robot keyword (i.e., step) and its corresponding status in the Context section of the Execution Details of the Generic Test.



References