YaST/Web/Development/Testing

From openSUSE

Conventions for testing

This document collects conventions for testing the code that conforms the WebYaST product.

Testing of Ruby on Rails Application

See A Guide to Testing Rails Applications for more details.

In short, automatic tests in Ruby on Rails can be divided to three levels:

• unit tests - model testing - low-level tests for the core functionality (see e.g. Unit Testing for more details)
• functional tests - controller testing - test one action
• integration tests - workflow testing - test sequence of several actions

WebYaST also uses UI tests using Selenium framework, see the UI Testing page.

Needed packages

In addition to standard Ruby on Rails packages these additional packages are needed for testing:

• rubygem-test-unit - base testing framework
• rubygem-mocha - library for mocking function calls (to avoid execution of the critical part on the current system and make the tests system independent)
• rubygem-rcov - for checking code coverage (how much code is covered/tested by the tests)

Starting Tests

Simply invoke rake command in a WebYaST directory to start all tests. The command will start all tests which belong to the current directory, if the no test is found it searches in parent directory.

Use test:units, test:functionals or test:integration rake target to start only tests of the requested type (e.g. call rake test:units to run only unit tests). Use rake test TEST=foo_test.rb to start only one test.

WebYaST RPM packages do not contain tests, they are only available in the GIT repository. (The tests are continuously run in the integration server and they are not needed at production server.)

Code Coverage

Use rake test:test:rcov to run all tests with coverage support. At the end it will generate a HTML coverage report and print a link to it. Open the link in a web browser and check the coverage. The main page of the report contains a summary, you can click on a file to see the lines which are not covered by tests. Ideally, all files should have 100% coverage.

Note: neither 100% code coverage guarantees that the code is bug free and perfect! See RCov page for more details.

Writing a Test

Tests are stored in test/{unit|functional|integration} subdirectory depending on the type of the test. The test file name must match *_test.rb pattern to be automatically found by rake.

The following sections describe solving WebYaST specific problems.

YaST REST Service

Code in the REST service has objective to provide a RESTful resource representing data coming from the system.

This data can be retrieved by various ways:

• PackageKit
• YaST YaPI calls via DBus
• Calling command line programs (over DBus SCR execute service, to preserve policies)
• Others

Conventions

Don't access the system

Your tests can't depend on what the system returns. Use the Mocha library to mock the result of classes and test your code against different system scenarios you consider.

Model encapsulation

If you have a UserController which retrieves data from the system users, encapsulate the way the data is retrieved by providing a small ActiveRecord/ActiveResource like API ( User.find, User.save ), this gives you the following advantages:

• The controller looks like any Rails controller
• Testing the controller without going to the system requires you to mock the User class data which is simpler than mocking the YaPI calls.
• You need to test any layer of abstraction you create, so you need to test the YaPI calls anyway.

For example, if you have a SoftwareService resource class, which reads the product files in /etc/zypp/services.d and exposes them as a RESTful resource, you could create an index controller like:

def show
  # get all files
  Dir.glob("/etc/zypp/services.d/*").each ...
  # parse ini file, make a hash object
  hash = INI.parse(files)
  render hash.to_xml
end

You could test the controller in the controller test class by stubbing what Dir returns when called with that directory, and what File reads when called with those paths.

Dir.stubs(:glob).with("/etc/zypp/services.d/*).returns(filelist)
# stub File.read to return a hardcoded ini file
...

See how it gets complicated? You really don't want that in the controller test. Because the controller test should be focused in the workflow and not in grabbing the data. You should test how the controller *reacts* to certain data and exceptions. Instead, create a SoftwareService model, with the typical find, save, etc methods, and allow to construct it from its attributes.

Then in the controller class, mock

  s1 = SoftwareService.new(attrs)
  s2 = SoftwareService.new(attrs)
  SoftwareService.stubs(:find_all).returns([s1, s2])

And now you can test your controller without worrying how the data is actually read. You can also mock SoftwareService to throw exceptions on certain values and see how your controller reacts.

Then you need to create a new test for the SoftwareService class. In this test you can mock the Dir class to return a fixed filename list when Dir.glob is called with the services.d path.

Then you need to make the decision if you mock the File class to return something fix when constructed with one of those fixed paths. For example you can mock File.new to return an StringIO object constructed from some fixed INI-style string (which acts like a normal file, as it also inherits IO), but only when called with the fixed paths you know.

Dir.stubs(:glob).with("/etc/zypp/services.d/*").returns([ "/etc/zypp/services.d/service1.service", "/etc/zypp/services.d/service2.service"])
Dir.glob("/etc/zypp/services.d/*")
=> ["/etc/zypp/services.d/service1.service", "/etc/zypp/services.d/service2.service"]

The file does not exist, so any attempt to open it will fail.

File.new("/etc/zypp/services.d/service2.service")
Errno::ENOENT: No such file or directory - /etc/zypp/services.d/service2.service

We can stub File.new to return a String based IO object, if we know that our code will only do operations like read.

File.stubs(:new).with("/etc/zypp/services.d/service2.service").returns(StringIO.new("[service]\nattr1=val1\n"))

Now you can read the file even if it does not exist, you will be dealing with a StringIO object, constructed from a fixed string we want to test with:

f = File.new("/etc/zypp/services.d/service2.service")
=> #<StringIO:0x7fa223f10a28>
f.read
=> "[service]\nattr1=val1\n"

But then we will need to mock File for every SoftwareService method that reads the files. Instead, you can move the reading code to a separate method content_for(path), mock that one to return a fixed INI String when testing any method of SoftwareService, and additionally add a test to test content_for(path) that will be the only test mocking the file. See the pattern?

• You have various methods that to be tested require to mock something complex, like ComplexClass
• You move the code depending on File to a separate method read_data (fictional name) which talks to ComplexClass
• You test all methods by mocking the read_data method to return a simple String, Array, etc.
• You add an additional test to the testcase that tests the new read_data method, and only here you mock ComplexClass

That way you simplify your testcase without reducing coverage.

YaST Web Client

The Web Client part has similar problems as the Web Service part regarding to automatic testing.

Do not connect to a Web Service

The Web Client also should not access the system - in this case it means it should not use a real Web Service (REST) service. There are two possibilities:

• Mock the REST system calls
• Fake REST responses using e.g. FakeWeb plugin

Here is an example of mocking REST calls:

  # override the WebYaST proxy
  class Proxy
    attr_accessor :result, :permissions, :timeout
    def find
      return result
    end
  end

  # a part of the result
  class Action
    attr_accessor :my_attribute

    def initialize
        @active = false
    end
  end

  # REST result
  class Result
    attr_accessor :foo, :bar

    def fill
        @foo = Action.new
        @bar = Action.new
    end

    def save
      return true
    end
  end

  def setup
    # create a testing request
    @request = ActionController::TestRequest.new

    # create a REST result and fill it with fake data
    @result = Result.new
    @result.fill

    # create a WebYaST proxy 
    @proxy = Proxy.new
    @proxy.permissions = { :read => true, :write => true }
    @proxy.result = @result

    # register the proxy fake
    YaST::ServiceResource.stubs(:proxy_for).with('org.opensuse.foo_servce').returns(@proxy)

    # create the controller for testing
    @controller = FooController.new
    # skip the login check
    FooController.any_instance.stubs(:login_required)
  end

  # here define the tests
  ...