Stories

back Stories Overview

User Experience at SAP

Projects

ASUG

Conference Reports

Future Scope

 

CHI 2005 – Spotting the Trends ...

By Gerd Waloszek, SAP User Experience, SAP AG – May 25, 2005

In this report, I present a few personal notes from the CHI 2005 conference, which took place in Portland, Oregon from April 02 - 07, 2005. As this was my sixth CHI conference, I apologize for not being as enthusiastic as many of my colleagues whom I talked with about it. There is definitely a difference between whether you listen to an interesting presentation for the first or for a third or fourth time. My comments maybe somewhat biased due to my previous CHI and HCI experiences.

CHI 2005 logo

On the other hand, I attend the CHI conferences to find out how the HCI field changes over time and whether we experience "progress" or not. Each CHI conference that I visited marked a specific point in time and helped me to spot trends and to integrate them into my personal picture – or puzzle – of our still emerging field. Apart from the pure HCI trends, there are also organizational and business trends that cannot be overlooked. Currently there are heated debates on:

  • Cost-justifying usability using measures, such as ROI (return on investment) or TCO (total cost of ownership)
  • Reducing costs for usability activities by outsourcing and offshoring
  • The growing pace of company reorganizations and how they affect the HCI profession
  • The self-image of the HCI community as a profession (science, engineering discipline, craft, art ...)

As the CHI 2005 conference reflected these topics, I will focus on them in my report.

 

Outsourcing and Offshoring

Certain changes go unnoticed even though you know that they are going on. One of these is outsourcing usability work. Currently, the vast majority of it is outsourced to India, followed by China, Russia, Canada, Ireland, and Mexico. Whether outsourcing pays off, is still an open question. A cost factor of nine for outsourcing has been claimed: You can hire three people at a third of the cost offshore. Other sources claim that the economical gain is only marginal – and will become increasingly less attractive in the future because the problems can be immense and the wages are rising "offshore." Thus, often it is more important to be close to relevant markets than saving money by outsourcing.

Panelists - Outsourcing

The panelists (center: Jon Innes, SAP Labs, Palo Alto)

Companies, such as Human Factors International (HFI) or Computer Associates (CA) have been successfully practicing offshoring usability work for years already. But to be successful, you need a deep understanding of the cultural differences and of which tasks can be easily outsourced and which cannot. Typically, routine tasks can be outsourced while more open-ended or conceptual tasks cannot.

 

Is ROI an Effective Approach for Persuading Decision-Makers of the Value of User-Centered Design?

To me, this panel looked a bit like a put-up affair: Three of the four panelists seemed to agree in beforehand that ROI (return on investment) should be dismissed as a valid measure. Solely Dennis Wixon from Microsoft's game division put forth the ROI case. Actually, he put forth the case of his RITE (rapid interactive and evaluative testing) method. His contribution would have been better called "RITE – A Commercial," instead of "Strategic ROI – A Commercial."

Panelists - ROI

The Panelists (left: Dan Rosenberg, SAP AG)

While IBM's Jennifer Lai, who stood in for Clare-Marie Karat, claimed that she never had to justify user-centered design using ROI statements, David Siegel characterized ROI as a tactical measure that undermines risky business decisions, such as introducing new technologies or moving into new business sectors. He contrasted it with strategic thinking, which has a long-range focus and treats risk as inherent. In his opinion, ROI can be potentially self-destructive for the whole HCI profession because of its shortsightedness.

SAP's Dan Rosenberg repeated his story of the myths that ROI is based on and offered TCO (total cost of ownership) as an alternative measure – among others, because he, too, regards TCO as strategic. TCO was introduced by the Gartner group. It recalls the "ecological point balance," and is plagued in my opinion with a bunch of problems of its own. For example, if you look at a TCO model it is hard to find the places, where usability comes in – which may lead some people (such as managers) to conclude that it isn't important at all. And with both, ROI and TCO, you are never sure whether you can really attribute certain outcomes to certain actions, such as the effort invested in a user-centered design process.

 

Corporate Re-Orgs: Poison or Catalyst to HCI?

I do not believe that this panel really answered the question "Poison or Catalyst to HCI?" Instead, we got more personal (typical American?) recommendations, such as: Do a good job, be flexible, and so on. Re-orgs come from "heaven" (that is, from the board) and you had better adapt to them and try to make the best of it for yourself in order to survive. Kelly Brown from eBay at least hinted at the fact that there might be a contradiction between business (or company) and professional goals. Years ago, Don Norman proclaimed that usability people need to "infiltrate" the management in order to support their profession. I would like to add: this is probably the most promising way to ensure re-orgs, which are catalysts for HCI.

Panelists - reorgs

The panelists

 

The Great Debate: Can Usability Scale Up?

What can you expect from such a debate? I did not know Eric Schaffer from HFI but I have known Jared Spool since 1998. So, I expected some sort of "show event" – and, to a certain degree, it was. I learned that Jared Spool can cite wrong numbers, be accused of this by someone from the audience, but in the end nobody cares. Somehow, this is symptomatic of the current state of the profession because it touches on the question of what type of discipline usability is: user interface design, user experience, or whatever label you put on it? In my article "User Interface Design – Is it a Science, an Art, or a Craft?", I came up with the conclusion that it's a craft, and this is what Jared in the end also was trying to make clear. So, even despite the shakiness of Jared's arguments, I am basically "with him."

Eric Schaffer and Jared Spool

The debaters: Eric Schaffer (left) and Jared Spool (right)

As a physicist who moved into the realms of psychology, I had to recognize that there is a difference between "hard" and "soft" sciences. So, I am always skeptical whenever someone tries to make us believe that UI methodology can lead to predictable, reliable results (in the end expressed in numbers, such as ROI or TCO). As evidence, Jared cited Rolf Molich's CUE4 investigation, in which Molich showed that contrary to the Nielsen-Landauer "law," the more errors were found in the user interface, the more teams tested it. And more disturbingly: There was little overlap between the results of the different teams.

According to Jared, "craft is based on the skills of individuals, while engineering tries to achieve repeatable results independent of the engineer's talents." I also agree with his statement: "Evidence suggests that usability practice is more craft than engineering, but we sell it as an engineering practice." Eric Schaffer and others do this selling and make good money with it. But in my opinion, we are still in the "age of the gurus." People, such as Karen Holtzblatt, Alan Cooper, Jakob Nielsen, or Bruce Tognazzini – and I would also add Jared Spool to this list – support my hypothesis by declaring that solely their own approach is the right one (in science, there is – I know I am generalizing a bit – only approach, the scientific). At least, I am in line with Eric Schaffer when he sees the importance of gurus diminishing in the future.

Debate - A question

A tough question for the debaters ...

Let me conclude this section with three of Jared Spool's introductory statements:

  • High investment in user-centered design doesn't result in more usable designs
  • In fact, it usually results in less usable design
  • Usability practice doesn't scale up (Eric believes that it is ready to scale up right now)

Generally, I agree with him. But the correct conclusion is not to not invest in user-centered design (UCD) or only halfheartedly. In my opinion, Jared just described the current state of affairs, which reflects an insufficient integration of UCD into the development process as well as most software companies' general view of what is important and what is not.

 

Looking Forward by Looking Back: Early Gestural Interfaces for Live Electronic Music Composing and Performance

The closing plenary was introduced by a minute's silence in commemoration of Jef Raskin, the "father" of the Macintosh user interface, who passed away in February.

Michel Waisvisz demonstrates one of his musical instruments

Michel Waisvisz demonstrates one of his musical instruments

The main act was a presentation by Michel Michel Waisvisz, who reported the history of his self-built electronic music instruments, which are commanded by gestures and buttons. And, of course, he demonstrated some of them. Waisvisz' musical installations can also be used in group settings, or even distributed over different locations.

So, what is the point of such a presentation at an HCI conference – apart from its entertainment value? After having reviewed a couple of books on design, especially on the relationship between technology and user experience, my answer would be the following: Designers and artists highlight the fact that human experience is not a "prefabricated" thing that you can design into something by using certain techniques – as some UI companies and experts claim. Instead, they claim that people are creative and use their creativity to make up their own, open-ended experiences. It is the open-endedness of human-experience that the designers strive to remind the HCI community of again and again – an idea that they feel seems to have been forgotten by this community.

 

References

Outsourcing

ROI

  • Wixon, D. (2005). Is ROI an Effective Approach for Persuading Decisionmakers on the Value of User-Centered Design? Panel Discussion at Conference on Human Factors in Computing Systems (CHI 2005).
    Download Slides (2.6 MB)
  • Games User Research at Microsoft Game Studios: www.mgsuserresearch.com
  • Daniel Rosenberg, Seven myths of usability ROI: www.baychi.org/calendar/20031014

Debate

Closing Plenary

 

All photos by Gerd Waloszek (taken with Minolta Dimage A200).

 

top top