UX Installer Test Outline

From openSUSE

Goals we have identified for this test:

1.Determine if the Subject understands the install process.

-Installer? Partitioner? Network settings?

2.If they don't understand, how much will they experiment before they ask for help?

-What would the OS have to do for the Subject for that Subject to continue using the "experimental" OS after an initial evaluation period?

3.At what point does the Subject feel that the install process is complete?

-What metrics do they use to identify that "completed" status.

1. We envision a five-part test procedure:

A.Standard pre-interview verbal questions (see descriptions of these sections here: Installer_Test_Procedure)

B.Install process (first virutal machine), with three or four tasks or questions (e.g. \ please change the hardware configuration somehow, or can you identify the partition arrangement?)

C.Questionnaire regarding installation metrics: primarily multiple choice questions (good for generating statistical measurements) with a few short-answer/essay type questions. I realize that I have not formulated the questions below in this style, but this is just a first pass.

D.First boot experience (second virtual machine) with potential to explore the OS environment or possibly show them a website comparing screen shots of various installers.

E.Standard test debriefing/recap, to include time for Subject to present their ideas for improving the product.

2. Installer test based on roll-back images

Using (probably VMware or other roll-back capable image builder) some image of the [SLED, openSUSE?] install process, create a role for the subjects to play (as Eva suggested... you just got introduced to linux, and you want to install it at home...) and assign them tasks which are designed to allow us to draw conclusions about the process. Please see goals of the test above. I'm concerned that if we encourage the Subjects to make drastic changes during the install process, those changes will not be affected upon initiation of the second portion of the test, after package installation.

3. Two sub-tests? Novices vs. Experts

We discussed possibly running two tests, one for novices (home users) one for experts (professional users, i.e. IT personnel), each consisting of three parts. Re: Eva's idea of running 20 novice users, I think there is substantial value in testing (say 30% of our total pool of Subjects) expert users who have installed an operating system in the past: these users will likely be able to express their desires for an installer more clearly and have a greater understanding of the process. Our database of past users will come in handy for identifying these experts, as well as down-selecting from internet responses (name@hotmail.com~=novice, name@xyz.org or .net~=expert)