Systems Evaluation

Having systems evaluations skills means you are able to identify measures or indicators of system performance and the actions needed to improve or correct performance, relative to the goals of the system.

Systems evaluation methods are used to evaluate the utility, usability, and learnability of a system design. These methods aim to assess how easy a particular system is to learn and use, and how well the system supports the tasks that the user wishes to perform.

Systems evaluation task examples

• Demonstrate that systems meet performance criteria.

• Compare two systems to find which performs better.

• Measure what parts of the system or workload cause the system to perform badly.

Systems evaluation methods

Heuristic Evaluation

A technique for assessing the usability of a computer interface that uses ten rules of thumb, such as "speak the user’s language", "provide feedback", "be consistent", and "provide good error messages". In a heuristic evaluation, the analyst evaluates how well the proposed interface follows the rules of thumb and provides feedback as to how it could be improved.

Walk-throughs/Cognitive Walk-throughs/Talk-throughs

In walk-throughs and talk-throughs, workers who knows the system perform a task using an actual system or a realistic mock-up for analysis. When performing a talk-through, the user is removed from realistic surroundings and merely verbalises the demonstration. Cognitive walk-throughs attempt to evaluate the state of the user's thought processes at each step of task performance, with emphasis on identifying aspects of the interface that are confusing.

Formal Usability Studies

Workers are observed or videotaped while they perform tasks using a proposed system in a controlled environment. They may also be instructed to think aloud when performing the tasks. By observing workers performing the same tasks under such controlled conditions, aspects of the interface which need improvement can be identified.

Rapid Prototyping

A prototype of a proposed system is presented to workers for critical comments. Revisions are made to the original prototype, producing a second version that is again presented to users for critical analysis. The process of revising and submitting to users continues, until criteria for acceptability are agreed upon.

Storyboarding

Storyboards are non-functional pictures or display screens of design concepts and how they would operate in a simulated scenario. Storyboarding enables system developers to receive user input and feedback early in the concept development phase.

Further Learning

• Online tutorials on systems evaluation are available at Systems Analysis and Design (Freetutes).

This content comes from The University of Essex