YaST/Web/Development/UI Testing

From openSUSE

What is UI testing and why we should use it?

UI testing is web application testing in a real web browser. The advantage is that it's possible to check the real behavior and it is possible to check some parts which are very hard to test using standard test cases - like javascript or AJAX functionality.

UI testing can check the web application in several browsers - checking the application in many browsers can be easily automated.

Emulating YaST-REST-Service

In order to test the UI without any environment like a corresponding REST-service (from which data will be received) we can use the REST emulation sinatra.

A framework has already been defined in

  web-client/webclient/test/dummy-host

Use the following command in order to start the REST-service:

  ruby host.rb

The access of the REST-service will be then

  http://localhost:4567

and can be used for login, logout, permission, language, time, patch, status, security, user ( to be continued ) UI-module. This feature provides a simple way for testing the UI especially while changing stylesheets, CSS, ....

Selenium Framework

WebYaST uses Selenium framework for UI testing. Selenium has several components, the most important are:

• Selenium IDE - A plugin for Firefox browser, it can record a browser session and replay it or export it as a test case
• SeleniumRC - Remote control component, provides a proxy server and contains bindings to several programming languages (like Ruby, Python,...)
• SeleniumCore - Set of helpers (Javascripts) which are injected into tested page, they simulate user's actions. This component is part of the proxy server in SeleniumRC.

How to run the WebYaST UI tests

Prerequisities

For running Selenium tests in WebYaST you need these packages installed:

• a package providing JRE (Java Runtime Environment) - like java-*-openjdk - the Selenium proxy server is written in Java and needs a JVM for running
• rubygem-test-unit package which contains test-unit ruby gem - this is an improved version of standard Test::Unit module shipped with Ruby
• rubygem-selenium-client package with selenium-client ruby gem - this package contains Ruby bindings to Selenium
• a supported web browser - currently only Firefox is supported

Running tests

Just execute

 rake test:ui

command in web-client/webclient directory in GIT repository. The test:ui task starts the Selenium server component then it starts a web browser (Firefox) and runs the tests in it. (A new browser is opened for each test suite.)

At the end it shuts the browser and the Selenium server down and prints the results of the tests.

How to write UI tests

The most simple way to write a test case is to use the Selenium IDE plugin. Here is short overview, see Selenium IDE documentation for more details.

Selenium IDE

Install the plugin from the download page.

In Firefox use Tools -> Selenium IDE menu item to start it.

Recording a session

Press the red button in the top right corner (by default it should be already pressed).

Then use the browser to record the intended test. The Selenium IDE plugin will record all actions like form filling, button pressing etc.

When the test case is finished press (deactivate) the red button to stop recording.

Replaying the test

Now you can replay the recorded test and verify that it works correctly. For replaying use the green buttons in the tool bar.

Adding checks

(Here is just an overview, see the documentation for more details.)

The recorded session does not contain any test, it contains just actions.

The tests should be added after the page is loaded (after clickAndWait command). Locate the command in the table, select the command below it (the new command are added before the selected item) and select Insert New Command from the context menu (right-click).

Now write some assert command into Command field. Selenium supports autocompletion, just write assert and select an appropriate command from the list. The detailed description of the command is displayed at the bottom.

The most useful asserts are probably:

• assertTitle - check the title of the page
• assertElementPresent - check whether a particular element is present somewhere on the page
• assertValue - check the value of an input filed
• assertTextPresent - check attribute of an element
• assertLocation - check the absolute URL of the page (e.g. after redirection)

Use assertNot* commands to assert a negative result.

Adding inline Ruby code

The WebYaST output formatter (see the next paragraph) supports inline Ruby code written in comments (with @RUBY: prefix). Use it for debugging calls or for more advanced tests. (Of course, the inline commands will not be executed by Selenium IDE, use them only if really needed.)

Integration to WebYaST

Add Ruby WebYaST output plugin to Selenium IDE

WebYaST defines base class Selenium::TestCase for all UI tests, the exported tests would have to be modified to this class. To automate this there is a custom output formatter for Selenoim IDE which exports the test case in WebYaST format.

Use Options->Options...->Formats->Add menu item in Selenium IDE to add the plugin.

Enter Ruby - WebYaST to format name filed and replace the format definition by ruby-webyast.js file. Use the latest version from GIT repository.

The best way is to open the file in the browser and copy & paste it into the text area. Confirm changes, now the Selenium IDE is able to export tests directly in WebYaST format.

Saving/exporting a test case

When the test is ready use save test cases and the test suite to XML files (native Selenium IDE format).

Use File->Export Test Suite As...->Ruby - WebYaST menu item to export the test as a Ruby script. Use *_test.rb file name scheme, otherwise rake test:ui task will not find it.

See the example how to use the saved files.

Example test suite

1. Starting the the webclient and webservice
   1. YaST webclient: web-client/webclient/script/server -p 4568</tt>
   2. REST service emulation: ruby web-client/webclient/test/dummy-host/host.rb

1. Simple test case - target host selection test
   1. Start WebYaST and open http://localhost:4568 in Firefox browser.
   2. Logout from WebYaST if you are already logged in
   3. Start Selenium IDE plugin (Tools->Selenium IDE)
   4. Add open command with target /
   5. Add assertElementPresent command with target //div[@class='services-list dialoge-wrapper width-65'] (check for div element with class services-list dialoge-wrapper width-65)
   6. Press Find button to check that the element was found (it should be highlighted)
   7. Stop recording
   8. Replay the test case, check that it works correctly
2. Advanced test case - login page test
   1. Use File->New Test Case to add a new test case
   2. Double click Untitled 2 test case to make it active
   3. Goto http://localhost:4568 web page
   4. Start recording
   5. Click on http://localhost:4567 link (Normally it should have the name dummy-host)
   6. Stop recording
   7. Add two test commands, check for Login and Password input fields:
      Command: assertElementPresent, target login (check for element with id=login)
      Command: assertElementPresent, target password(check for element with id=password)
      You can press Find to highlight the selected target.
   8. Replay and verify the test case
3. Save and export the complete test suite
   1. Save Untitled 2 test case to login_test_credentials.xml (use File->Save Test Case As...)
   2. Save Untitled test case to login_test_services_list.xml
   3. Save the test suite to login_testsuite.xml (test suite contains only references to test cases, use File->Save Test Suite As...)
   4. Export the test suite in Ruby - WebYaST format to login_test.rb file (File->Export Test Suite As...->Ruby - WebYaST)
4. Automatic testing
   1. Copy *.xml files to web-client/webclient/test/ui/src directory, copy the exported *.rb. script to web-client/webclient/test/ui directory in GIT repository checkout
      Note: the *.xml files are authoritative, if you need to change a test then load it into Selenium IDE and reexport it again to *.rb file.
   2. Run rake tests:ui to run all tests or run e.g. rake test:ui TEST=ui/login_test.rb to run a particular test
      While running rake tests:ui a YaST-webclient server will be started on port 4568 and a YaST-webservice emulation will be started on port 4567.
   3. If automatic testing passes without problems then push the new files to the GIT repository

Links

Here you can find Selenium documentation how to design tests.

This document describes how to test pages with AJAX scripts.