YaST/Development/New Control Center/Card Sort Study

From openSUSE

Summary and Discussion of Results

Work in progress

Some results at a glance

Important Demands - no order

Sorting and presentation

New sort order with merge of modules (e.g. AppArmor, User & Groups)

Pdf-file with mockups showing ~ 50 icons for the modules

• Some modules are merged
• Color, fonts, e.g. is not final!

Possible solution for discussion: new dialog page for client-server modules can reduce the amount of module icons

                   NIS is used as an EXAMPLE only
                    to demonstrate the principle
                           behind 
 

                         [Icon] NIS
                              |
                              |
                     Which type of system
                          is running?
                              |
                              |
          +-------------------+-----------------------+
          |                   |                       |
          V                   V                       V

    Which one do you      NIS Client              NIS Server  
    want to configure?
    ( ) NIS Client
    ( ) NIS Server

Look

* Note: My thinking about a better YCC produced a design similar to the new SLED YaST Control Center. This, before I saw the new SLED design!

Consistency Problem:

• The YCC design should be consistent to the Control Center of the Desktop in the background.

First Idea

The order of the groups at the left side needs some thinking 

+--------------------------------------------------------------------------------------+
| Filter               |                                                               |
|  [                ]  |  [Icon] Name                 [Icon] Name                      |
|  [ ] Display only    |                                                               | Just an idea
|      basic setup     |                                                               |
|                      |                                                               |
| Groups               |  [Icon] Name                 [Icon] Name                      |
|  My Favorites        |                                      +---------------------------------+
|  Recently Used       |                                      | This is the tooltip with a      |
|  All                 |                                      | concise but informative         |
|                      |                                      | explanation to describe the     |
|  Group 1             |                                      | explanation to describe the     |
|  Group 1             |                                      | function of this module. Incl.  |
|  Group 1             |                                      | some examples.                  |
|  Group 1             |                                      +---------------------------------+
|  Group 1             |                                                               |
|  Group 1             |                                                               |
|  Group 1             |                                                               |
|                      |                                                               |
|                      |                                                               |
|                      |                                                               |
| -------------------- |                                                               |
| Help*                |                                                               |   * It should be possible to provide
|  YCC Online Manual   |                                                               |     a detailed help text describing
|  YaST at openSUSE.org|                                                               |     the YCC
|                      |                                                               |
|                      |                                                               |
|                      |                                                               |
+----------------------+---------------------------------------------------------------+

Important Demands Concerning the Wording and Search/Filter

• Better use of consistent wording
  • Name of the Module should be short an show a consistent system behind. A style guide would be nice
• Enhanced explanatory tooltip
  • With better information about the functionality of the module
• Quality check of the translations
  • The German translation shows not the same quality as the English original
• The Filter/Search should be based on a kind of thesaurus
  • e.g. "Cups" will also display the printer module

Sub-Sections

Resources

Links concerning cardsorting:

• Collection of links
• Nielsen's Alert Box - How many users do you need?

Study Design

Sample Description

The subject sample contains of

• 29 expert users (27 male, 2 female)
• and currently 3 desktop users (2 male, 1 female) -- they will be handled as singular cases at the moment

Used Card Set

Remarks

• The cards showed the available YaST Modules after the openSUSE 10.2 default installation with one local user
• Three cards have been moved from the SaX to the set: VNC, Touchscreen, Digitizer Tablet
• The AppArmor section was mixed with the other cards

The 77 cards in alphabetical order:

• /etc/sysconfig Editor, Add Profile Wizard, Add-on-Product, AppArmor Control Panel, AppArmor Reports, Autoinstallation, Bluetooth, Boot Loader, Boot or Rescue Floppy, Date and Time, Delete Profile, Disk Controller, DNS and Hostname, DSL, Edit Profile, Firewall, Graphics Card and Monitor, Group Management, Hardware Information, Hostnames, IDE DMA Modus, Infrared Device, Installation into Directory, Installation Source, iSCSI Initiator, ISDN, Joystick, Kerberos Client, Keyboard Layout, Language, LDAP Browser, LDAP Client, Local Security, LVM, Mail Transfer Agent, Manually Add Protocol, Media Check, Modem, Mouse Model, Network Card, Netzwork Services (xinetd), NFS Client, NIS Client, NTP Configuration, Online Update, Online Update Configuration, Partitioner, Patch CD Update, Post a Support Query, Power Management, Powertweak, Printer, Profile Manager, Proxy, Release Notes, Remote Administration, Routing, Samba Server, Scanner, Software Management, Sound, Sudo, System Backup, System Restoration, System Services (Runlevel), System Settings, System Update, Digitizer Tablet, Touchscreen, TV Card, Update Profile Wizard, User Management, Vendor Driver CD, View Start-up Log, View System Log, VNC, Windows Domain Membership

Study Workflow

 +-----------------------------+
 | Phase 1                     |
 +-----------------------------+
 | o Welcome                   |
 | o Preliminary instructions  |
 | o General questions         |  Gender, age, profession, expert status, use of YaST
 +-----------------------------+
                |
                V
 +-----------------------------+
 | Phase 2                     |
 +-----------------------------+
 | o Instruction               |
 | o Cardsort                  |  Hand written records of subject's comments
 |   procedure with think      |  Sorting results saved by photos
 |   aloud method              |
 | o Question: Favorite        |
 |   modules                   |  Favorite modules saved by photo
 | o Optional question: Server |  Only 3 subjects
 |   modules                   |
 +-----------------------------+
               |
               V
 +-----------------------------+
 | Phase 3                     |
 +-----------------------------+
 | o Concluding Interview      |  Questions:
 +-----------------------------+  1) Tell some positive and negative aspects concerning 
                                     YaST Control Center
                                  2) What's about the module names and the descriptions 
                                     texts, what do you think about the quality? Can you give
                                     a school grade?
                                  3) Do you have some wishes concerning the YaST Control Center?
                                  4) In a general view, what do you wish from the YaST developers?

Illustrative Materials

Cardsort result. The outcome was saved via photos.

Selected favorite modules. The instruction was: "Think you have an already installed openSUSE 10.2 at home. Your machine is well configured. Which of the modules would you need in the YaST Control Center. Select zero to 20 cards."

Video capture -- the "think aloud" procedure was saved via low quality mpg videos

Results - Expert Group (N = 28 to 30)

Cluster Tree

The cluster tree in the background of the illustration was build by xSort (see http://www.ipragma.com/xsort for deatils) with different statistic options

• The difference is, the modules Modem, DSL, ISDN and Network Card have been moved to group Hardware in the first model and to group Network in the second model.
  

Expert group (N=29), Used cluster statistic options: average linkage plus flatten groups

Comments

• (1) Modules Language, Date and Time, and Keyboard Layout have been sortet to group System -- Desktop users perhaps would create a section with Personal Settings
• (2) Even experts did wrong here. The Profile Manager is used by laptop users to switch easy between different networks. The modules Add Profile Wizard and Delete Profile Wizard are AppArmor modules. All three modules are not connected to the User Management module.
• (3) Power Management and iSCSI Initiator have a lot of distance to the other items
• (4) In this calculation model, the experts moved the ISDN, DSL, Modem and Network Card module to the section Hardware
• (5) The probability to get two sections dividing Server Roles vs. Client Roles with a different card set containing Server Modules is very high
• 6) This section shows three sub sections, but the distances are very low in comparison to the other groups

The following cluster tree used different statistic options.

Expert group (N=29), Used cluster statistic options: average linkage plus use sub-groups

Comments

• (1) The probability to get two sections dividing Server Roles vs. Client Roles with a different card set containing Server Modules is very high
• (2) In this calculation model, the experts moved the ISDN, DSL, Modem, and Network Card module to the section Network
• (3) Experts sorted wrong here. The Profile Manager is used by laptop users to switch easy between different networks. The modules Add Profile Wizard and Delete Profile Wizard are AppArmor modules. All three modules are not connected to the User Management module.
• (4) Some AppArmor modules have a small distance -- this are the cards where the text contains the term "AppArmor"
• (5) (1) Modules Language, Date and Time, and Keyboard Layout have been sortet to group System -- Desktop users perhaps would create a section with Personal Settings
• (6) Power Management and iSCSI Connector show the greatest distance within this group
• (7) The functionality of the module Boor or Rescue Floppy was often unclear
• (8) Software is the section with the lowest distance between the items

Manipulation of Card Set

Some of the subjects manipulated the given card set

• There was no explicit order within the instruction to do this nor to avoid this.
• Observer reaction during study when a questions occured was: "do as you like it, the order should fit to your needs".

Illustration 1

Think aloud notes

Modules commented with "Function is not clear to me"

Some modules had unclear functionality to the 28 expert users. Modules with comments > 10 % of sample size:

• 10 users
  • Update Profile Wizard
• 9 users
  • Powertweak, Add Profile Wizard
• 8 users
  • iSCSI Initiator
• 7 users
  • Sudo, AppArmor Reports, System Settings, AppArmor Control Panel, Edit Profile, LDAP Browser, Delete Profile
• 6 users
  • Profile Manager
• 5 users
  • VNC
• 3 users
  • Installation into Directory, Kerberos Client, Media Check

Modules commented with "This is a bad name or description"

The 28 expert users often criticized the names or descriptions of the modules. The following list displays the quantity of comments by modules with N > 10% of sample size:

• 8 users
  • Hostnames
• 5 users
  • DNS and Hostname, Profile Manager
• 4 users
  • Update Profile Wizard, Add Profile Wizard, Media Check, Routing, Remote Administration, AppArmor Control Panel, Powertweak, VNC, Mail Transfer Agent
• 3 users
  • Proxy, System Settings, Autoinstallation, Local Security

Modules where user was unsure to choose a group

The following list displays the quantity of comments for modules with N > 10% of sample size where the subject had problems to choose one group.

• 4 users
  • NTP Configuration
• 3 users
  • Post a Support Query, VNC, Firewall, Keyboard Layout, Add Profile Wizard

Favorite Modules for Already Configured System

Instruction after the cardsort procedure:

• "Imagine you own an already installed and configured running openSUSE 10.2 at home on a normal machine.
• Which modules would you need to maintain your system.
• Please select a maximum number of 20 modules."

Results for Expert Group

• 54 cards had been selected by the 28 subjects, 23 cards not.
• In average, the 28 expert subjects selected 11 cards (SD = 5.7, minimum = 2, maximum = 21) -- Maximum of 20 is exceeded by the fact, that some subjects had merged cards

Illustration 1

Modules with selections <= 4

• Modules with 4 selections:
  • DNS- and Hostname, System Update, System Backup, System Restoration, etc/sysconfig.
• Modules with 3 selections:
  • LVM, Scanner, Hardware-Info
• Modules with 2 selections:
  • Boot-/Rescue-Floppy, Windows-Domain-Membership, ISDN, Keyboard Layout, NTP-Config., View System Log, NIS-Client, Mouse, Media Check, Network Services (xinetd), TV-Card
• Modules with 1 selection:
  • Startup-Log, Sudo, Release Notes, Routing, IDE-DMA-Mode, Mail-Transfer-Agent, Modem, AppArmor Control Panel, IRDA, Keyboard-Layout, LDAP-Client, Joystick, System-Settings, Add-On-Product

Modules without selection

• ... TBD ...

Illustration 2

Concluding Interview

Question 1 -- Positive and negative comments

Free translation: "Please tell some good and some bad points concerning the YaST, 1-3 points for each category would be fine.

47 Positive comments

Illustration 1

62 negative comments

Illustration 1

Question 2a -- Quality of the module names

Free translation: "Please tell me you opinion about the module names, are they usable, do you have remarks, can you give a school grade?"

Results

• Values:
  • N= 29
  • Mean= 2,41 (German system: satisfactory to good; 1 is best score, 6 is worst score)
  • SD = 1,35
  • Min =1
  • Max = 5
• Recorded comments:
  • No consistency - For me OK, excepting outliers - inconsistent - often for „normal“ people hard to understand - sometimtes bad, some are taskbased, some are themebased - there are outliers & inconsistent - I have no problem with it - all in all ok - exception AppArmor - inconsistency - AppArmor = 5 - some could be better - not clear enough - some are fine, some bad - some are bad - despite of any outliers - it seems that someone had problems to translate the module names - very accurate wording - generally ok - generally good - a few are gaga - good without AppArmor - some things are not self-explanatory - some outliers

Question 2b -- Quality of the module descriptions

Free translation: "Please tell me you opinion about the module descriptions, are they usable, do you have remarks, can you give a school grade?"

Results

• Values
  • N = 27
  • Mean = 2,94 (German system: satisfactory; 1 is best score, 6 ist worst score)
  • SD = 1,55
  • Min = 2
  • Max = 6
• Recorded comments:
  • bad, not all, but many - sometimes bad German - mostly ok, the redundant descriptions are bad - partly they are totally daffy, redundant - sometimes you don`t know whats the meaning is - is ok - present it a little bit more noticeable - all in all good - could be better, avoid redundancies - enhancements are possible - a lot of redundancy - could be much better - often bad - generally uninspired - in 2/3 you get no helpful information - mostly redundant - what I can see here is ok - someting is redundant, AppArmor is bad - thing which have poor names have mostly poor explanations too - explanation is of identical to the module name - you cannot describe it clearly with a picture

Question 3 -- YaST Control Center Wishes

Free translation: "What features should be implemented in the next YCC version?

Illustration 1

Detailed list

• 37.93 % (11) -- Sorting according user type
  • Different possibilities to sort according expert mode (21) (31) (32)
  • Expert/novice-mode (04) (08) (09) (11) (15) (18)
  • Seperate modules wich can destroy your system if you use them incorrectly (31)
  • Sorting for power users, all in one tab (29)
• 34.48 % (6) -- Possibility to generate favourites
  • The menue should be configurable (possibility to create favourites or new groups) (29) (07) (08) (13) (16)
  • I want to have easy access to the modules I use regulary (03)
  • A tab or something similar with my favourite modules (22) (32)
  • System can generate favourites (18) (16)
• 24.14 % (7) -- Sort according frequency
  • Software section should be sorted by frequency of use (24) (27) (28) (07) (22)
  • Often used modules should be in the front (32) (05)
• 24.14 % (7) -- Sorting according module dependencies
  • Sort modules within groups according their dependencies to each another (24) (25) (26) (29) (30) (31) (04)
• 24,14 % (7) -- Implement better search
  • An intelligent search function is important (21) (05) (11)
  • A feature like the hotspot function of Apple OXS would be nice (21)
  • I want to click fast through the sections (17)
  • A search function should reflect language (e.g. in Japanese there are different words for printer...) (27)
  • If you type in an name, YaST should display a list with tree and subgroups (02)
• 6.9 % (2) -- Skipp "Miscellaneous" section in current YCC
  • There is no need for that point (34) (22)
• 3.45 % (1) -- Sort alphabetically
  • The modules should be sorted alphabetically (28)
• 3.45 % (1) -- Sorting according system type
  • Sort modules according use of system (desktop vs. server) (15)

• (no count) -- Workflow, controllability
  • Modules should start in the right frame, not in a pop-up window (33)
  • It would be nice if I can control the GUI completely and comfortable with the keyboard (27)
  • The YaST should be more intuitive to handle (26) (30)
  • I' d like to use language-independent short-cuts to start the modules and within the modules (27)
  • There should be no root restriction to make simple settings like sound & keyboard (04)
  • YCC should behave like the control center of MacOSX (10)
  • Work with tabs like in MacOS. Mac is homogeneous. (14)
• (no count) -- Design, theme
  • Users should get to the modules over generic start icons, related to the respective theme (26)

Question 4 -- General Wishes Concerning YaST

Free translation: "You have 3 free wishes and the YaST developers will make them come true, please tell me ..."

Illustration 1