The dilemma of interaction design and faceless interaction.

Just saw this article about some new Apple patent. It is interesting how it moves toward a situation of new interactive modes, and particularly interaction without an interface. This is something that Lars-Erik Janlert and I have written about and discussed in some detail in a recent article.

Janlert, L-E., & Stolterman, E. (forthcoming). Faceless Interaction - a conceptual examination of the notion of interface: past, present and future. In the journal Human-Computer Interaction.

In the article we present a historical overview of how the interface has been understood over time. We also discuss some issues that interaction design is facing today, such as shrinking devices (less space for an interface) but increasing functionality (more this to do). We examine different ways this dilemma is handled today. And we specifically examine the idea of getting rid of the interface (as in the Apple patent above). We also discuss new issues such as cluttering which will become the next big interaction problem.


This is the abstract from the article

"In the middle of the present struggle to keep interaction complexity in check as artifact complexity continues to rise and the technical possibilities to interact multiply, the notion of interface is scrutinized. First, a limited number of previous interpretations or thought styles of the notion are identified and discussed. This serves as a framework for an analysis of the current situation with regard to complexity, control, and interaction, leading to a realization of the crucial role of surface in contemporary understanding of interaction. The potential of faceless interaction, interaction that transcends traditional reliance on surfaces, is then examined and discussed, liberating possibilities as well as complicating effects and dangers are pointed out, ending with a sketch of a possibly emerging new thought style."

What makes a prototype novel?

At the NordiCHI  Conference in October, Mikael Wiberg and I presented a paper:

Wiberg, M. & Stolterman, E. (2014) What Makes a Prototype Novel? – A Knowledge Contribution Concern for Interaction Design Research. NordiCHI '14, October 26 - 30 2014, Helsinki, Finland.

The paper is about something anyone engaged in HCI research is familiar with. Here is how we present it in the introduction of the paper:

"Every time we review a paper describing a new interactive system, every time we go to a conference or when we are presented with a new interactive system from industry we repeatedly find ourselves asking “Have I not seen this system before?” or stating “This system does not remind me of anything I have ever seen...”. These questions are related to a fundamental research consideration concerning how it is possible to conceptualize and relate different designs to each other and to the existing body of knowledge. In short, we address the question “what makes a prototype novel?” and accordingly a manifestation of something new, a knowledge contribution to our field. "

This is both a simple and difficult question. How do we know that a new design actually represents a new knowledge contribution or not? Most of us in the field agree that prototypes are crucial as part of the knowledge production process, but in what way? In the paper we discuss some of the earlier attempts that have been made to solve this question. We also propose some potential ways to move forward in a more structured and formal way. We propose the notion of generic design thinking or concepts. 

Below is the Discussion section from the paper. Of course it is not easy to understand just the final section of a paper, but it may give a sense of what we are trying to do. If you are interested in these questions, let us know.

"DISCUSSION – WHAT IS A ‘NEW’ DESIGN? AND HOW CAN WE ADVANCE HCI DESIGN RESEARCH THROUGH GENERIC DESIGN THINKING?

When generic design concepts are used in architecture, there are two ways of handling designs that are new: as novel or as unique. Importantly, for something to be considered new it is not sufficient merely to be novel in the sense of having “odd properties”. Instead, for a design to be unique or new, it must involve at least one of the following criteria: 

• the application of an established generic model to a new problem or in a new domain 

• a design that combines elements from multiple established generic models 

• the addition of a new element to a known generic model manifested in a design 

• a combination of a new generic model and a design that defines a new design space such that the design demonstrates the potential scope of the new space. 

In this context, novelty that stems from an evolution of a design’s underlying model reconfigures the landscape of design spaces; if done particularly well, it creates new space within this landscape that others can join and exploit. 

We see several important implications of this suggestion for the advancement of HCI design research. 

First, generic design thinking reject designs that are not properly situated within a web of existing and already known design ideas. The new cannot be advanced without understanding how it relates to existing design ideas. That is to say, a new ultimate particular (a concrete design) needs to be anchored in the general (that is, in some theoretically articulated idea). 

Secondly, generic design thinking implies a shift in focus away from specific properties of a given ultimate particular towards generic dimensions in new designs. This shift has implications for what we need to express with a particular design. It also raises questions about which factors should be incorporated into a design and which can be omitted when designing prototypes during the research process. This could potentially reduce the difficulty of developing research prototypes as fully implemented systems (and the need to include a lot of specific system features, etc). 

Thirdly, generic design thinking implies the need for more deliberate work in HCI on the formulation of classes of interactions. Today, direct manipulation, embodied modes of interaction, and agent-based interaction models could be seen as some relatively stable classes that are important for the formulation of generic design principles in HCI. But what other kinds and ways of grouping interactions and interaction technologies can we imagine? And how can we move forward and become more specific? And what are the existing good examples? 

Finally, generic design thinking provides a practical tool to improve our ability to compare and evaluate different designs. In this way, it could provide a foundation from which to address design quality and to make judgments about designs that are rooted in more than just the opinion of an individual designer. This aligns well with the proposed concept of interaction criticism [2]. 

In this paper we have proposed, described and exemplified generic design thinking in the format of a four-step method and approach to systematically move forward (design) while also more systematically understand and learn (analyze) from past designs. Although we have so far only described this as a first draft of a method we are convinced that this approach might redirect our field slightly from being heavily future-oriented to also acknowledge the utility of working backwards from a design to its conceptual roots – to trace design ideas through the analysis of designs. Importantly, generic design approaches require critical analysis of the history of design within HCI in order to anchor the new and novel in the history of ideas. 

In wrapping up our paper, we should return to its basic message. We do have recent research stressing the importance of concept-driven design research [25] and we do have a good understanding of how ‘strong concepts’ [13] can advance our field. At the same time we lack methods to systematically relate different concepts to each other and in relation to particular designs. Here is where generic design thinking can play an important role as to systematically advance our field while keeping our design-driven approach.  Given this take on the subject we should state that in order to answer the most central question for design-driven HCI “when is a new design a knowledge contribution?”, we must first, as a field of research, establish the method and approach for guiding the systematic work of conceptualizing and theorizing these designs. In this paper we have suggested ‘generic design thinking’ as an initial attempt to move in the direction of the development of one such method and approach."

The Anatomy of Prototypes

In preparation of a lecture on Monday I had to re-read an article that I wrote together with Youn-Kyoung Lim and Josh Tenenberg called "The Anatomy of Prototypes: Prototypes as Filters, Prototypes as Manifestations of Design Ideas".

I don't think I have read the whole article in a few years which is always an exciting  and a bit nervous experience. Anyway, this time I was pleasantly surprised. I really find this article to be interesting and still useful. Unfortunately I have not done any more work on this topic since we wrote this article but after having read it now, I think I  have to.

I am more convinced than ever that what our field needs are analytical tools that makes it possible to investigate specific entities in a structured way. In this article we propose an anatomy of prototypes that supports such investigations of one of the most important entities in our field--the prototype.

It would be great if we could develop similar "anatomies" for other entities in our field. The 'power' of anatomies (or frameworks) is that they (if done well)  provide clear definitions that support examinations, categorizations and understanding without being prescriptive. The anatomy of prototypes that we provide in our article is a conceptual tool for the analysis and description of every possible prototype without saying anything at all how to design prototypes or what constitute a good prototype (except in a very abstract but principled and precise way).

Anyway, good to return to "old" texts and to discover that they still can be valuable and not outdated.

Faceless Interaction - a conceptual examination of the notion of interface: past, present and future

Now the article by Lars-Erik Janlert and me is published on the Human-Computer Interaction Journals website. I am very happy to see this article published!


Janlert, L-E., & Stolterman, E. (2014). Faceless Interaction - a conceptual examination of the notion of interface: past, present and future. In Human-Computer Interaction. 

Abstract

In the middle of the present struggle to keep interaction complexity in check as artifact complexity continues to rise and the technical possibilities to interact multiply, the notion of interface is scrutinized. First, a limited number of previous interpretations or thought styles of the notion are identified and discussed. This serves as a framework for an analysis of the current situation with regard to complexity, control, and interaction, leading to a realization of the crucial role of surface in contemporary understanding of interaction. The potential of faceless interaction, interaction that transcends traditional reliance on surfaces, is then examined and discussed, liberating possibilities as well as complicating effects and dangers are pointed out, ending with a sketch of a possibly emerging new thought style.

Book note: "Ways of Knowing in HCI" by Judith S. Olson and Wendy A. Kellogg

A new book just arrived in my mailbox. It is "Ways of Knowing in HCI" by Judith S. Olson and Wendy A. Kellogg. I have spent some time today going through the different chapters. I read some and browsed others. Overall I find this to an excellent contribution to the field of HCI research. The edited book contains 18 chapters, each describing a particular research approach, method or technique commonly used in HCI research.

Of course, most of these methods have been described elsewhere and in most cases also in more detail and depth, but the unique aspect of this book is that all chapters are written by HCI researchers who can relate each method to HCI research and also give examples of where it has been used, what it mean to use it in HCI research, what constitute a good job when using a particular method.

I fully agree with the Editors in their Epilogue when they state that we should read all chapters and not just the ones about methods we already are familiar with. I followed their advice and I found right away wonderful argumentations, explanations and critiques of methods that I am less used to. This reading influenced me right away and made me more favorable of them. I am impressed by the quality of the chapters, even though I find some chapters not reaching the level of what could be accepted in a collection like this.

To me this book is already a required reading for all PhD students in the field. I can not see any reason why any graduating PhD student would not be familiar with all these methods, at least at the level of knowing about them as presented here.

I am looking forward to see some reviews of this book (more in-depth analysis than my note here :-) to learn more about what is not covered in the book, what is not done well enough, etc. In the name of developing a discipline we need all members of the HCI research community to engage in a book like this. We need a discussion and debate about "ways of knowing in HCI" and this book creates a great platform for such a discourse.

I really want to thank the Editors and the authors of this book for taking the time to develop and write these chapters. This is not the kind of work that most scholars see as being the most important and rewarding, but it is necessary work. So, thanks for all the great work!

A growing problem in HCI research

After being to CHI and another gathering of HCI related people in the last few weeks, I have to share one reflection. I am not sure that my observation and definitely not my interpretation is correct or not, so comments are welcome.

The observation is that the field of HCI research is growing in size but also in scope. What today is considered to be HCI research spans a far wider area than every before. HCI today includes research that traditionally might have been seen as engineering (for instance, design and development of devices and systems with the purpose to explore technical possibilities), it also includes research that traditionally would be seen as within the social sciences or behavioral sciences (the latter has of course always have been part of HCI), but also research that can be considered humanistic or cultural studies, political or global studies. The research has also expanded in scope when it comes to application areas such as education, health care, transportation, entertainment, sustainability, etc.

This expansion of the field is of course exciting and has led to new perspectives and new knowledge that has enriched HCI. However, at the same time I am concerned by this development. It is possible to see this development as an expansion that leaves an empty space in the middle.

I would argue that a large part of todays HCI research could and maybe should be seen as research in other disciplines. For instance, research related to education should be evaluated and published in educational research venues and contribute to that field unless there is also a serious contribution to the core of HCI. This of course raises the question of what is the core of HCI.

If research in HCI do not in any sense contribute to our understand of human computer interaction in some general or universal sense, and if it is only an application of what we already know in yet another field, then it may be a contribution to that application field but not to HCI.  So, if someone applies HCI theory and knowledge (whatever that is) in another field to explore and examine a phenomena without bringing back some serious insights to HCI theory and knowledge then it is not HCI research.

What this type of expansion leads to is unfortunately in many cases research that do not contribute in a serious way to the core of HCI while also being questionable research in relation to what is the standard in the "other" field. If the research really contributed to those other fields then the research should be evaluated and published in those fields.

Ok, I understand that this argument raises a lot of issues especially around the notion of what is the core of HCI research but I see it as important for our field to discuss those issues if we want to be able to produce knowledge contributions that are distinct and valuable in relation to the contributions from other disciplines. HCI research is not going to be successful or recognized by how it is doing research or by how it is able to "use" knowledge from other fields, it will only be successful if there is a core knowledge contribution that is of a kind that no other field really cares about or produces knowledge about. HCI research will not become successful by expanding the field, not by approaching and including more application areas and topics. HCI research will only be successful if we can offer something valuable at the core that constitute well developed knowledge that no other discipline has done or will do, and that can be valued by its own merits.