Human Performance at the Computer – Part 3: Perceived Performance

By Gerd Waloszek, SAP User Experience, SAP AG – November 13, 2008 • original article (design tidbit)

In this series of four articles I would like to investigate the topic of human performance at the computer, introduce new insights, and discuss performance-related issues that can be addressed by UI designers – including the concept of perceived performance. In the current article, I would like to focus on perceived performance, take a look at the factors that influence how users perceive the system's and their own performance, and present approaches to improving perceived performance.

 

What is Perceived Performance?

The term perceived performance refers to the observation that users tend to judge the performance of a computer system subjectively and often perceive it differently from its objective performance. The (preliminary) Wikipedia definition of perceived performance is as follows, and includes some examples:

  • Perceived performance, in computer engineering, refers to how quickly a software feature appears to perform its task. The concept applies mainly to user acceptance aspects.
    The amount of time an application takes to start up, or a file to download, is not changed by showing a startup screen or a file progress dialog box. However, these do satisfy some human needs: They make the application appear faster to the user as well as providing a visual cue to let them know the system is handling their request (slightly changed for better understandability).

When discussing perceived performance, two main aspects are considered. Firstly, if there is a difference between objective performance and its subjective judgment or perception by the user we can ask what causes the difference. I will investigate this aspect in the following section. Secondly, we can ask how we can improve perceived performance. The respective approaches are discussed further down.

Judging Perceived Performance

When searching the literature, I found many references to instances where users perceived performance to be better after certain changes had been made, even though these changes had made the objective performance (slightly) worse. Such reports must, however, be taken with a grain of salt, because they typically refer to the overall process. For example, the authors may compare different conditions and observe how much time a Web page needs to load completely under each condition. If a Web page loads incrementally, it may indeed take longer to load completely, compared with displaying a blank page first and then showing the page content nearly instantaneously. On the other hand, when a page loads incrementally, users can start work – reading in particular – earlier, can get sooner oriented to a page sooner, and may thus also finish their work earlier – provided that page elements that load later are not needed at the beginning. Here, performance-oriented guidelines might come in and advise UI designers how to arrange the page content so that the respective usage scenario is supported and the users' efficiency improved (see Human Performance at the Computer – Part 4: On the Way to Performance-Oriented UI Guidelines). To give an example, a corresponding rule might state: Display important information first.

As the example of incremental page load demonstrates, human performance at the computer can neither be discussed from a technical nor from a UI design perspective alone, because the interpretation of outcomes may depend on which aspect you focus on. From a technical perspective, it definitely makes sense to compare the complete page loading times under both sets of conditions. However, when looking at the man-computer system as a whole, the task level would provide the right granularity, and the time that is needed to complete a task or task step would be the focus. In my opinion, it would actually be misleading to focus solely on loading times for complete pages, because this would obscure the fact that incremental page load allows users to get oriented sooner and to start work earlier, both of which contribute to faster task completion times.

 

Influences on the Users' Perception of Performance

To understand the differences between objective performance and its subjective perception by the user, we need to explore the factors that have an impact on the users' perception of performance. I searched the literature, found a number of factors, and placed them in a common framework. Some factors relate to the users' perception and knowledge, some to the task at hand and how users perform it, and others to the overall impression of the system.

User Perception and Knowledge

On the one hand, perceived performance depends on:

  • What users see
    • From the users' current experience of their work at the computer (observation, perception)
  • What users know
    • From the users' prior experience of their work at the computer (expectations): People have established expectations based on their past experiences of the time required to complete a given task. Relevant aspects are the task's complexity and the users' familiarity with the task.
    • From system feedback (information): The system can provide different kinds of feedback that informs users about a delay and – even better – its cause and expected duration.

Thus, the perception of performance depends on direct observation of the system (perception), expectations from past experiences (memory), and the information that the system provides users with (feedback). In addition, perceived performance also depends on the individual's tolerance of delays (personality; Shneiderman & Plaisant, 2004).

Task

Perceived performance also depends on task-related aspects such as:

  • How soon users can continue with their work
    • How soon they can start reading
    • How soon they can start interacting (clicking, typing, …)
  • How successful users are in completing their tasks

Thus, users do not want to be impeded by the system and, of course, they want to accomplish their tasks successfully. They also base their judgment of the system's performance on these criteria.

Appropriateness

Furthermore, perceived performance depends on:

  • How well the interaction conforms to behaviors in the real world
    • Continuity: smooth movements, appearance/disappearance of objects, transitions, …
    • Consistency: consistent behavior, variation and predictability of waiting times, ...

Thus, users prefer smooth and consistent system behavior to abrupt and erratic behavior and judge the system's performance accordingly. I will call this factor preliminarily appropriateness.

 

Approaches to Improving Perceived Performance

The list above lends itself to suggestions for improving perceived performance. Further approaches listed below are taken from the literature. Note that many suggestions are listed redundantly because they can be assigned to several aspects. Moreover, a lot of them can also be regarded as methods for improving the responsiveness of a computer system – the two aspects of responsiveness and perceived performance are often hard to keep separate (but we're not in a science class, here...).

Reducing and Hiding Waiting Times

One approach to improving perceived performance is to reduce users' waiting times in various ways, for example, by not requiring users to wait until a process has finished, or by discarding processing steps that are no longer needed. Masking waiting times usually results in a shift in the time or priority with which a processing step is handled by the system.

  • Reducing perceived waiting times: Often, there is no need to make users wait until a process has completed before they can continue their work. Waiting times can also be reduced by cleaning or reordering the event queue.
    Examples: Load pages incrementally, flush obsolete events from the event queue, change the priorities of events in the event queue.
  • Masking or hiding waiting times from users: Certain user requests and book-keeping tasks can be delayed and/or transferred to background tasks or parallel processing threads.
    Examples: Perform tasks in the background, delay them, or perform them during idle times.
  • Both aspects can be combined by using more advanced processing strategies. There are static and dynamic approaches to supplying users with a fast preliminary result and performing time-consuming calculations at a later point in time.
    Examples: Fake heavyweight calculations (show fake first, show final result later), adapt the processing strategy in order to keep up with the user (trade in quality for quantity).

Let Users Continue or Start their Work as Soon as Possible

This approach is related to the previous one and focuses on the work aspect instead of on the delay. It is also based on the premise that users often need not wait until a process has finished. The following recommendations refer to two user interface aspects: (1) the presentation of results and (2) the readiness for user input.

  • Provide incomplete results: Often, it is not necessary to make users wait until a process, such as loading page or displaying a large image, has finished.
    Examples: Load pages incrementally (users can start reading), display coarse versions of images, show empty tables.
  • Do not block user input: Often, it is not necessary to block all user input. It may be permissible for users to scroll or interact in other ways with an application, to abort it, or to switch to another one. Generally, it is recommended to at least allow users to abort processes, particularly if they are long-running, and start new ones.
    Examples: Accept certain types of user input (such as scrolling or canceling of actions) to allow users to abort an application or long-running process or to switch to another one.
  • Reduce start-up time: Reducing the start-up time of operating systems and applications allows users to start to work earlier (see also design issues).
    Examples: Reduce the OS start- up time by saving a memory image to disk when users log off: never shut down (used in mobile computers) except when really necessary.

Put the User in Control: Allow User Input and Provide Options

The following recommendations refer to the control aspect and are based on the observation that users are more tolerant of waiting times when they are in control.

  • Put the user in control – users are more tolerant when they are in control:
    • Be responsive during delays: Do not block user input during delays: accept at least certain types of user input.
    • Allow users to abort long-running processes or the application and start new ones.
  • Provide choices – having choices makes users feel that they are in control:
    • Free users to do other things while they are waiting for a process to complete, for example, allow them to carry on with other tasks.
    • Allow users to abort the current task and to start another one.
    • Give users the option to choose between faster performance and better functionality.

Improve User Tolerance of Waiting Times

In many cases, waiting times cannot be reduced or even eliminated, and we need to ask how waiting can be made more tolerable for users.

  • Provide (timely) feedback: If users are informed about the cause and duration of a delay, they usually accept it and adjust their expectations accordingly.
    This topic is discussed at length in a separate series of article (Waiting at the Computer: Busy Indicators and System Feedback Part 1, 2, 3).
  • Put the user in control: Users are generally more tolerant of waiting when they are in control.
  • Do not force users to be idle: Allow them to carry on with other tasks while they wait for a process to complete, or allow them to abort and start another process. If a user interface behaves in this way, users regard it as responsive and tend not to think of the application as slow.

Ensure That Users Are Successful

As already mentioned, the way on which users perform a task also has an impact on how they perceive a system's performance.

  • According to Perfetti & Landesman (2001), users perceive a system as performing faster if they can complete their tasks successfully.

Consistency

Consistency seems to help reduce user stress:

  • Inconsistent system behavior causes unpredictable performance, which is known to stress users. Making the system perform consistently should remove this source of stress. However, a certain variation in response times is acceptable for users.
  • Provide plausible waiting times that correlate with specific user actions (in contrast with erratic waiting times)
  • Variations in response times: Modest variations in response times (50% more or less than the mean) appear to be tolerable and to have little effect on the users' performance. Frustration emerges only if delays are unusually long (at least twice the anticipated time).

"Tricks" for Waiting Times

The following tricks help reduce the effect of waiting times on users:

  • Include delays to make applications appear "smoother" (for example, when users open a window or menu – typically done at OS level).
    Example: Let windows and menus open and close smoothly, not instantly.
  • A few long response times leave a greater impression on the memory than a large number of shorter ones.
    Example: Split up long-running processes – such as loading large amounts of data into a table/page – into several smaller ones.
  • Provide plausible waiting times that correlate with specific user actions (in contrast to erratic ones).

Design Issues

  • Start-up time: According to Shneiderman & Plaisant (2004), rapid start-up is an important design issue. Users get annoyed if they have to wait several minutes for a laptop or a digital camera to be ready for use. Consequently, fast starts are a strong distinguishing feature in consumer electronics.
    BTW: In a recent New York Times article, Richtel and Vance discuss recent attempts to cut down the starting time of operating systems.
  • Trade-off between rapid start-up and rapid usage: Shneiderman & Plaisant (2004) mention this as a related issue. Rapid start-up often implies that usage is slowed down because some data or functions have to be loaded at usage time. A general recommendation is hard to make, but it is evident that rapid start-up is more important for a digital camera (which is a computer, too...) or a hand-held computer than for a desktop or laptop computer.

 

Conclusions

In this article, I present factors that influence how users perceive the performance of a computer system and list approaches to improving perceived performance – some of which might also be regarded as approaches to improving system responsiveness. We meet the paradox that making perceived performance better may actually lead to worse objective performance, although the interpretation of the results may depend on which aspect of performance is being considered. Anyway, there is a reason for accepting small degradations in system performance: Good perceived performance can lead to higher user motivation and thus to better overall performance. In addition, if users can continue their work sooner or abort unwanted processes, this not only increases perceived performance but also their overall performance, although only some so-called "perceived performance" measures have been taken. I do not want to engage in hair-splitting discussions here, however. The primary goal is to improve the performance of the overall man-computer system, irrespective of whether we may regard our methods as ones for improving the system's responsiveness or its perceived performance.

Perceived performance includes the users' tolerance of waiting times. Tolerance can be increased by putting the user in control and also by applying "perceived performance tricks," some of which take the human need for orientation and information into account. One such example, although one might debate whether it should be regarded as a "trick" or not, is to provide timely and appropriate feedback. This topic is, however, beyond the scope of the current article and is therefore discussed at length in a separate series of article (Waiting at the Computer: Busy Indicators and System Feedback Part 1, 2, 3).

 

References

 

top top