Conventions for testing

This document collects conventions for testing the code that conforms the WebYaST product.

Testing of Ruby on Rails Application

See A Guide to Testing Rails Applications for more details.

In short, automatic tests in Ruby on Rails can be divided to three levels:

unit tests - model testing - low-level tests for the core functionality (see e.g. Unit Testing for more details)
functional tests - controller testing - test one action
integration tests - workflow testing - test sequence of several actions

Needed packages

In addition to standard Ruby on Rails packages these additional packages are needed for testing:

rubygem-test-unit - base testing framework
rubygem-mocha - library for mocking function calls (to avoid execution of the critical part on the current system and make the tests system independent)
rubygem-rcov - for checking code coverage (how much code is covered/tested by the tests)

Starting Tests

Simply invoke rake command in a WebYaST directory to start all tests. The command will start all tests which belong to the current directory, if the no test is found it searches in parent directory.

Use test:units, test:functionals or test:integration rake target to start only tests of the requested type (e.g. call rake test:units to run only unit tests). Use rake test TEST=foo_test.rb to start only one test.

WebYaST RPM packages do not contain tests, they are only available in the GIT repository. (The tests are continuously run in the integration server and they are not needed at production server.)

Code Coverage

Use rake test:test:rcov to run all tests with coverage support. At the end it will generate a HTML coverage report and print a link to it. Open the link in a web browser and check the coverage. The main page of the report contains a summary, you can click on a file to see the lines which are not covered by tests. Ideally, all files should have 100% coverage.

Note: neither 100% code coverage guarantees that the code is bug free and perfect! See RCov page for more details.

Writing a Test

Tests are stored in test/{unit|functional|integration} subdirectory depending on the type of the test. The test file name must match *_test.rb pattern to be automatically found by rake.

The following sections describe solving WebYaST specific problems.

YaST REST Service

Code in the REST service has objective to provide a RESTful resource representing data coming from the system.

This data can be retrieved by various ways:

D-Bus (See here for simulating D-Bus calls for testing)
PackageKit (See here for simulating PackageKit calls for testing)
YaST YaPI calls via D-Bus
Calling command line programs (over D-Bus SCR execute service, to preserve policies)
Others

Conventions

Don't access the system

Your tests can't depend on what the system returns. Use the Mocha library to mock the result of classes and test your code against different system scenarios you consider.

Model encapsulation

If you have a UserController which retrieves data from the system users, encapsulate the way the data is retrieved by providing a small ActiveRecord/ActiveResource like API ( User.find, User.save ), this gives you the following advantages:

The controller looks like any Rails controller
Testing the controller without going to the system requires you to mock the User class data which is simpler than mocking the YaPI calls.
You need to test any layer of abstraction you create, so you need to test the YaPI calls anyway.

See here for an example.

YaST Web Client

The Web Client part has similar problems as the Web Service part regarding to automatic testing.

Do not connect to a Web Service

The Web Client also should not access the system - in this case it means it should not use a real Web Service (REST) service. There are two possibilities:

Mock the REST system calls
Fake REST responses using e.g. ActiveResource::HttpMock (~~FakeWeb~~ also works but does not have an RPM).

Here is an example of mocking REST calls.

openSUSE:WebYaST testsuite

Contents