Friday, September 28, 2018

Faceless Interaction: Interaction Richness and Precision

The growth of intelligent assistants of different kinds is leading to more faceless interactions as we have defined it in our book "Things That Keep Us Busy-the elements of interaction" (Janlert & Stolterman, MIT Press 2017). Below is an excerpt from the book that touches on what this change could mean when it comes to interaction richness and precision.

"Faceless interaction may seem to open up for smooth or analog interaction more than surface-bound interaction does, but perhaps that is a false impression. Spoken natural language has a dominant digital component in the phonemes, morphemes, words, phrases, sentences, and so on, although there remains an analog component as well in the form of intonation and prosody (but note that tone in itself can be used for digital communication as well, as in Mandarin where there are four distinctive, semantically significant “tones”). There are also important examples of analog surface- bound forms of expression, such as drawing and painting. The richness that is inherent in analog interaction usually comes at the cost of skill. The ease with which users can handle complex interactions via surface-bound digital expressions—for example, by clicking buttons without a lot of motor skill or precision required—is a strength that faceless interaction may lack. So, even though the richness of analog and faceless interaction is appealing it may also lead to lack of precision and ease.
       For instance, playing the electronic musical instrument called the theremin seems to be much harder than playing the violin. The completely free gestures by which the theremin is operated are hard to make precise. To enable precise actions, it seems that some kind of physical support and resistance is required to deliver precise haptic feedback, implying surface- bound interaction.
       Free gestures seem to leave room for endless variations and subtle nuances, also open for more spontaneous expressions and reactions. At the same time, they are, just as much as the vocal apparatus, a usable basis for creating digital expressions and impressions. Consider, for example, regular sign languages and smaller sign vocabularies developed for specific uses, for example, the hand and arm signs used by military personnel.
       Auto-Tune is an audio processor that can correct the pitch of singers sing- ing slightly out of tune, digitizing the pitch of an analog voice. Something similar could no doubt be arranged for the theremin. The use of Auto-Tune has been criticized both for making inept singers sound as if they actually “can sing”—and for flattening, “photoshopping,” the human voice. That you can sing off key shows there is richness of expression (expressive elbow room) but if you don’t develop your ability to control pitch you cannot exploit that richness. It is quite clear that small, tightly controlled deviations from the nominal pitch work as an important expressive device in vocal music, as in much instrumental music.
     It seems as if faceless interaction has a tendency to influence the relationship between richness and precision. Whether this relationship also has the quality of being a tradeoff is less clear. What is clear is that any designer making decisions about faceless interactions has to consider the relationship between richness and precision, between digital and analog, in a more developed way than is required for traditional interfaces." (Page 161-162)

Monday, September 24, 2018

Understanding Designing: Getting Too Close

There is a lot of research aimed at 'exploring', 'dissecting' and 'examining'  important human activities with the purpose to 'reveal' and 'unpack' its inner structure and mechanisms.  For instance, creativity, learning, and designing are all human activities that are constantly examined.

This ambition is easy to understand.  Humans want to improve their activities and to be able to improve they have to understand how things work. For instance, to improve designing, first, we need to have a good understanding of what it can do, how it is done, how it can be done, and maybe how it should be done, etc. So we need more research on the process of designing.

We can formulate abstract theories and philosophies about designing as a human activity. We can study what designers are actually doing when they design. We can produce infinite amounts of data on different design processes, in different areas, with different designers, with different purposes, knowledge, skills, experience, etc.

But, can we get too close? The knowledge about designing that can be created is infinite. We can go layer after layer, closer to the 'real' thing. But for each layer, we add combinatorial complexity that quickly becomes impossible to handle and manage. We end up with knowledge that is extraordinarily complex and as fast as it grows in detail and complexity, it loses its practical relevance and potential guidance for improvement.

This is a well-known effect in systems theory. Getting closer gives you a clear picture of a defined part of the whole but may lead to a completely wrong understanding of the whole. This is what I see a lot today when it comes to research about designing. We see an overwhelming stream of research results dealing with this or that aspect of designing. We learn a lot, but where is the broader picture. Where are the attempts to combine all the details, to develop a whole?

We might need the details...maybe, but we definitely need the understanding of the whole.

Tuesday, September 18, 2018

Book note: "Design for the Pluriverse - Radical Interdependence, Autonomy, and the Making of Worlds" by Arturo Escobar

There are many different kinds of books about design and designing. The way they differ is diverse. It is fascinating to see the breadth and scope of books trying to describe, prescribe, define, advocate, or change design.

A simple list (far from comprehensive) would include design-related books that engage with:
-- "how to" practical aspects of the design process (approaches, methods, tools, skills, etc),
--  how to relate design to other approaches, such as art or science,
--  the history of design,
--  designed artifacts and systems in a particular field (buildings, chairs, systems, pens, etc.)
--  philosophical and theoretical thoughts on design
--  ___________

There are of course others too. For instance, there is a category of design books that engage with design and designing form the perspective of what to achieve with design. These treatments of design are usually a bit more philosophical and definitely more ideological and political. In this kind of books, design is almost seen as a "tool", as a way to achieve a particular goal.

One such book is recently published and written by Arturo Escobar with the title "Design for the Pluriverse - Radical Interdependence, Autonomy, and the Making of Worlds".

This is a book that "presents a new vision of design theory and practice aimed at channeling design's world-making capacity towards ways of being and doing that are deeply attuned to justice and the Earth." (from the book cover).

In the description, we can see how design is seen as a "tool" as the author wants to use "design's world-making capacity" to achieve "being and doing that are deeply attuned to justice and the Earth". Escobar does this by relying on certain philosophical foundations and what he calls 'ontological design'. It is an interesting book and an ambitious project that Escobar is involved in.

To me, maybe the most interesting aspect of the book is not the proposed "new vision" but a question that Escobar asks. He writes "Here again we confront one of the key issues of this book: can design be extricated from its embeddedness in modernist unsustainable and defuturing practices and redirected toward other ontological commitments, practices, narratives, and performances? Moreover, could design become part of the tool kit for transitions toward the pluriverse?" (p 15)

This type of question is similar to what is often asked in science and art. Each approach is often accused to be the instrument for certain goals and disinterested in others. They are accused to have some 'built in' bias that leads the approach/tool to only support certain outcomes. Escobar is arguing that designing has a built-in bias in its "embeddedness in modernist unsustainable and defuturing practices". Escobar's ambition is to reveal that bias and to explore and develop a possible alternative 'bias'. As a project, I find this highly interesting.

To me, maybe the most interesting observation stimulated by this book is the question about the nature of the core of design as a human approach to change. Escobar must see that the built-in bias in design is not an intrinsic quality, not part of the core. If it were, then the project would be futile. Instead, the alternative 'bias' seems to be something that can be added on to or infused in the approach. So, is there in Escobar's view a design approach that is not biased, that is the pure approach? Or is it always the case that any use of the 'tool' design is instilled with some bias?

(Of course, we can see how this question relates to the old discussion in the philosophy of science about the question of science as unbiased or not.)

Wednesday, September 05, 2018

Want to think like a designer

I saw this headline today

"Want to think like a designer? Try these 4 simple exercises
Even simple drawing exercises can get your creative juices flowing."

Why is it that thinking like a designer seems to be so easy to do and to learn. We do not very often see headlines like

"Want to think like a heart surgeon? Try these 4 simple exercises
Even simple cutting exercises can get your surgical skills flowing."

Of course, there is a difference. We cannot train as surgeons by cutting in people, but all of us can make drawings without any dangerous outcomes. But that does not necessarily make us good designers or help us think like designers. To do good design is difficult. It requires a lot of knowledge and expertise and experience.

Tuesday, August 14, 2018

New course: Elements of interaction

I am very excited to start my new course on Monday next week.  It is a graduate course called "The elements of interaction". It is based on my book "Things that keep us busy--the elements of interaction" (Janlert & Stolterman, 2018). It is always exciting to start a course but even more when the course is new and you have never taught it before. It will be fun and hopefully work out well!

Friday, July 13, 2018

Interacting with versus being with

In our book "Things That Keep Us Busy - Elements of Interaction" we make the case that interaction is an activity that has to fulfill certain criteria to be called interaction. We define a "window of interaction" with eight dimensions. Each dimension has to be within a certain range for an activity to be understood or labeled as interaction. This means that not all activities where humans 'use' things or systems are interactions.

So, what is everything else, all our encounters with the reality around us, if not interactions? Well, most of the "interactions" with our everyday environment seem better defined as being with. For instance, it does not feel natural to say that we interact with our home (even though this is maybe changing today). Instead, we live in our home. We might interact with people, but that seems to be most appropriate when it is people in specific roles, we usually do not say we interact with our children, spouse, or friends (even though some might, since our use of language has changed a lot).

If we accept this distinction between "interacting with" (interactive things and systems) versus "being with" (family and friends) is it possible to say something about how our contemporary society is changing. Yes, I think so. It seems as if there is an ongoing movement that transforms "interacting with" to "being with". People are looking for interactions with things that are natural and intuitive, that are similar to what they are used to when it comes to "being with". We are not interacting with our thermostat anymore, we are living with a home that recognizes our presence and desires. A home that adapts to us and our habits, that lives with us. We are more and more living with our smartphones. We do not necessarily see it as if we are interacting with them. They are always present, they are our companions, they are part of who we are.

Some questions arise. What is it that we gain and lose when we move from "interacting with" to "being with"? What are the long-term consequences of such a shift? When should we strive to not make this shift happen? And when is such a shift a way to reach an everyday life that is more "natural" and less determined by technology?

Friday, July 06, 2018

Designers are not heroes

In the midst of today's glorifying of design as an approach that can achieve anything, there is an unflattering stroke of hubris. Of course, I truly believe in design as a unique approach that can deliver outcomes that set designing apart from other approaches. But when designers start to see themselves as the 'one', as those who will solve the issues that no one else can solve, then I have some issues.

Designers commonly work in service of a client. They are paid to do work for a client. This is all good. When designers act on their own, without a client, they become activists (or in some cases artists). This is also fine. To be an activist means that you primarily take a personal and often political stance.  It means that you act on some deep beliefs, values, and/or ideology. In that case, you are not a designer, you are an activist that maybe uses some form of a designerly approach.

This is quite certainly a non-issue for most people and maybe to me too. But there is a serious aspect to this. When people start to act in the world and claim that they do it in the name of "design", then I see a problem. It creates a very specific problem for the tradition or approach of design. Designers are not necessarily equipped to deal with all kinds of problems in the world, which means that they will fail. It is a similar kind of problem or hubris as when some scientists argue that science is the way to approach all societal problems.

It is obvious to me that design as an approach is not 'designed' to handle complex societal issues in absence of a client. Each stakeholder involved in an issue may use a designerly approach of course, but there is no 'pure' or clean designerly way of approaching an issue that would make designers more suited than others to deal with a societal problem (unless the problem is of a type that is uniquely suited for a designerly approach).

Maybe more problematic is when designers see themselves as saviors or even heroes that are equipped with a more powerful approach than others, and they start to 'help' others without working in a close client relationship. Client-less 'independent' designers are no different than any other type of activist trying to impose their 'solutions' on others. To be clear, I have no problem with designers doing this as activists. In many cases, great designers can achieve great results in this manner. But it should not be done in the name of 'design'. It should be done as someone aspiring to create change and who is trying to use a designerly approach.

I truly understand that this reasoning is subtle and to most has no value or importance, but for me it is crucial.

Wednesday, June 27, 2018

What if HCI became a fashion-driven discipline?

A few years ago I wrote a paper together with Yue Pan (at the time a PhD student) about HCI and fashion. The title was "What if HCI became a fashion-driven discipline?". I have since then been surprised that this paper has not been getting more attention since in my mind it is quite controversial and challenges the field of HCI in an interesting way.

Today, I do believe we are even further in a transition of HCI and interaction design into a fashion-driven discipline. Such a transition will or can have a huge impact on the field if taken seriously. Well, for now, it is not taken seriously. But I think we have to, so I am trying to push this paper.....

And here is the reference:
Pan, Yue, and Erik Stolterman. "What if HCI Becomes a Fashion Driven Discipline?." Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2015.

Monday, June 25, 2018

Opening Keynote at DIS 2018

I recently had the pleasure of giving the Opening Keynote at the ACM DIS 2018 conference in Hong Kong. It was my first visit to Hong Kong and only my second in Asia. The conference was really good. Good size and on average very good papers. I really enjoyed participating. And to experience Hong Kong was an extra bonus.

Here is the abstract for my Keynote presentation:

To Study Interaction and Interfaces: 
An Approach and Some Findings

It is hard to deny that our artifacts and environments are becoming more and more complex, more and more “alive,” and as a consequence more and more demanding. We have to interact more. There seems to be no retreat or escape from interactivity. Some well-informed critics worry that the proliferation of interactions and interactive things has already gone too far. Their concerns raise many questions. Does interactivity in fact increase? How can we know? What does it really mean to claim that it does? And if indeed it is increasing, what does it mean? And why is this happening? And should something be done? Despite this development there seems to be no precise idea of what interaction is and what being interactive means, beyond a vague notion that it is some kind of interplay, usually optimistically understood as good-natured cooperation.

In this talk I will present the work I have done for many years, together with my colleague Lars-Erik Janlert. As our approach, maybe best described as analytical and philosophical, we have examined properties and qualities of designed artifacts and systems; primarily those properties that are open for manipulation to designers, that is, properties that designers can and do intentionally affect by their design decisions (and thus in principle are possible to control). Rather than taking users and their subjective experiences of the artifacts and systems as the primary target for examination, unfashionable as it may be, we have chosen to be objective in the sense of focusing on the artifacts and systems. Apart from discussing our approach, I will briefly introduce some of our main results consisting of some developed definitions of existing (and some new) concepts, such as, interactivity, interactability, interactiveness. I will end with some comments on what this kind of investigation can tell us about the future by introducing the notions of faceless interaction, interactivity clutter, and interactivity fields.

Thursday, May 17, 2018

A talk in Oslo "Things That Keep Us Busy - the elements of interaction"

I am very happy to be invited to give a talk at Oslo University next Friday, May 25th. The topic of the talk is my new book "Things That Keep Us Busy - the elements of interaction"

Monday, April 30, 2018

New course: The Elements of Interaction

This coming Fall I will teach a new (primarily graduate) course that I will develop over the summer. It is called "The Elements of Interaction" and it is based on my new book "Things That Keep Us Busy--the elements of interaction" (MIT Press 2017). The book is co-authored with my colleague Lars-Erik Janlert.

I am quite excited about this new course and I wonder if there are any courses like it out there. If you know, please let me know! And if you are interested or have questions, just email me.

Here is a description of the courses (at least as I plan now):

The Elements of Interaction

We are surrounded by interactive devices, artifacts, and systems. The general assumption is that interactivity is good -- that it is a positive feature associated with being modern, efficient, fast, flexible, and in control. Yet there is no very precise idea of what interaction is and what interactivity means. In this course, we will investigate the elements of interaction and how they can be defined and measured. We will focus on interaction with digital artifacts and systems but draw inspiration from the broader, everyday sense of the word. We will explore how the interface has changed over time, from a surface with knobs and dials to clickable symbols to gestures to the absence of anything visible. We will examine properties and qualities of designed artifacts and systems, primarily those that are open for manipulation by designers, considering such topics as complexity, clutter, control, and the emergence of an expressive-impressive style of interaction. The course is based on the idea that understanding some basic concepts and terms of interactivity and interaction can support interaction design and inform discussions about the future of interactivity.

The course is directed at graduate students with an interest in human-computer interaction in any form. Students in the course will develop a fundamental understanding of interaction and interactivity and an ability to analyze how interactive devices and systems interact and influence their users and environment.

The course will also help and support students to imagine new forms of interactions and to support them in the design and develop of interactive artifacts and systems.

The course will cover:

• The everyday reality of interaction and interactivity
• Understanding interaction and interactivity
• Basic elements and definitions of interaction
• How to study, analyze and measure interaction
• Examination of interaction complexity and control, interactivity clutter, etc.
• Implications for interaction design and UX
• The future of interaction (faceless interaction, interactivity fields, etc.)

As the core reading of the course we will use the book: 

Things That Keep Us Busy – the elements of interaction” by Lars-Erik Janlert and Erik Stolterman (MIT Press, 2017)

Friday, April 27, 2018

A reflection after visiting CHI 2018 Montreal

Back home after a few days at CHI in Montreal. Overwhelming. Impressive in size, breadth and culture.

A lot can be said about HCI research, how it is changing, where it is moving, based on what happens at CHI. This is why I like to go to CHI. Not the individual papers or presentations, but the chance to get a sense of the whole field.

I made one new observation this year. I don't know if it is true or only a result of what I personally happened to listen to and see, but I will share it.

I experienced that the research presented at CHI seemed to drift and shift in two distinct ways. I saw more good design based research made by young design oriented researchers that take such an approach to research as given and as legitimate. I also saw more scientific research. People who study interactive technology and systems, but also different forms of interactivity, using traditional scientific methods and approaches. They did this not with the primary purpose of finding or developing new applications to real problems, that is, to support design, but with the purpose to develop a deeper understanding of certain aspects of human computer interactions.

I see these developments, both of them, as excellent for HCI research. For them to come together in one conference where they inform each other is the way it should be.

My personal critique of HCI research over the years has often been that it has been 'inbetween' research. It has not been real scientific research and it has not been real design oriented research. Being in between is the worst place to be as a researcher. It leads to confusion of purpose and of what the outcome is and how it can be evaluated. So, what I saw this year is positive and I hope it continues.

Wednesday, April 11, 2018

Studying the nature of interactivity (HCI)

I have always been drawn to research that is trying to understand the 'nature' of some phenomenon. There are several reasons for this. I am attracted to the idea that as a researcher I am actually improving my own understanding of the phenomenon I am studying and increasing my ability to explain its nature, and that this knowledge grows over time and is in some sense is cumulative. As a consequence of this, I have had two major research projects during my career. One has been to understand the nature of designing as a human approach to change and the other has been to understand the nature of the interaction between humans and interactive artifacts/systems.

I have realized that this aim is commonly confused with a radically different purpose which is to develop support that can improve peoples ability to design new technology and systems. As any kind of basic research, the results it produces will never be able to tell anyone what to design or create. Basic research primarily provides a deeper understanding of some phenomenon.  Instead of leading to prescriptive knowledge that could make design easier, it leads to increased complexity and from a design perspective often increased difficulty.

After being involved in basic research for many years I am more than ever convinced that it is not just necessary, it is also possible, even when it comes to a field like HCI. It is possible to study the nature of interactivity. It is possible to develop a deeper understanding of interactivity as a real phenomenon, something we, over time, can actually become better at explaining with sound and well-grounded theories and models.

For this to happen HCI research has to change some of its fundamental assumptions. For instance, HCI research has as a primary goal to lead to some form of (immediate) improvements or usefulness. In Habermas terms it is possible to say that HCI research is driven by a "control knowledge interest" and less of a knowledge interest of "understanding" or "emancipation" (which are his other two categories.  Of course, I am not saying that research with a "control" knowledge interest is not needed or should not be done. HCI is as a research field primarily aimed at improving practice so that is fine. But the field would benefit from also engaging in these other knowledge interests with a strong passion.

I am therefore both happy and proud of the work I have done with my colleague Lars-Erik Janlert about the nature of interactivity, where we did set out to do what I discussed above. We examined an existing phenomenon (interaction/interactivity) with the purpose to describe and explain its nature, to the best of our ability. Of course, we only reached so far. But I believe that it is a start that hopefully will inspire others. The result is the book "Things That Keep Us Busy -- the elements of interaction" (MIT Press, 2017)

Tuesday, March 27, 2018

One of the best books ever on design

Again I returned to the writings of David Pye. His writings have been with me since the early 80s. The nature and aesthetics of design" is one of the absolute best books ever written about design.

Monday, March 26, 2018

Interaction design, complexity, and virtuosity

One of the most preached principles in design, and particularly in interaction design, is to strive for simplicity. It is yet difficult to find any examinations of what simple really means when it comes to design (there are some good exceptions, such as Maeda and Mollerup, see references below).

In many cases, being simple is of course good. Nevertheless, we also know that we live in a world that is complex and sometimes requires complex actions. We also know that people can do amazing things even with devices that are highly complex. Virtuosity can be achieved. So, the question becomes, can we design artifacts that require complex actions in a way that could support the efforts of reaching virtuosity?

Below is an excerpt from our book "Things That Keep Us Busy - the elements of interaction" (MIT Press, 2018). This is from Chapter 6 "Control".

"6.5 Virtuosity

Can we imagine artifacts that are highly complex while still being inviting to a user and providing incentives for continuous engagement, maybe even spurring a few users to aim for extreme levels of mastery? Let us consider the violin example again. As we noted in chapter 5, the violin combines low internal and external complexity with high interaction complexity, which apparently invites a range of user behaviors. Most people initially find the violin extraordinarily difficult to interact with even though the artifact itself is quite simple: just a few strings mounted on a soundboard and a bow. Beginners are not able to make any real music on the violin even though they of course can make (terrible) sounds.

We also know that the violin invites virtuosity, a display of expertly handled extreme interaction complexity. Virtuosic violin performances have a long tradition and many famous composers have written music specially tailored to let distinguished players show off their virtuosity. When it comes to musical virtuosi, it is not uncommon to hear comments that virtuosity is more a technical achievement or circus act than an expression of musical insight and depth of interpretation. It may be that listeners, rather than being moved by the music, are impressed and awed by the display of almost superhuman skills.

Why is it that some artifacts seem to invite virtuosic use while others don’t? Are there examples of digital artifacts or systems that have the same virtuosity-inviting quality as the violin? There seem to be few parallels to the violin example: low external complexity yet still inviting virtuosity. We have to remember that up until recently digital artifacts and systems have typically had relatively high external complexity. The development of human–computer interaction has been dominated by a constant effort to refine and keep controllability on par with ever-expanding functionality, rather than any ambition to lower external complexity, not surprisingly resulting in externally complex artifacts and systems. Of course, lately with the proliferation of small digital artifacts and apps with specialized functionalities, increasingly under the pressure of the interface bottleneck
problem, external simplicity has become a key issue and mark of good design. However, these artifacts do not seem to invite virtuosity, perhaps because the interaction complexity usually is low, too low.

Another explanation of why we have not seen convincing examples of virtuosity with digital artifacts could be the up till now typically strongly discretized input and output and strict turn taking of digital artifacts, quite far from the analog and continuous flow of violin playing. However, discrete input combinations related to desired output combinations in a complex manner do exist, for instance in some computer games, which open for the possibility of some sort of virtuoso performances; similarly, computer hackers can make dazzling performances of rapidly finding and fixing software problems, hammering away at breakneck speed on a command-based interface. Even with more mundane examples, such as highly complex office software with huge numbers of commands, layers of functionality, we may be impressed by the brilliant technique the professional user displays. Such examples demonstrate fast decision making under pressure in a situation where there is a lot going on to keep track of. This could potentially be understood, and savored, as a kind of virtuosic performance.
A difference is that the interaction is overtly digital and the external complexity considerable: it is a form of “combinatory” virtuosity rather than the “smooth” virtuosity of top-class violin playing. A violin seems to allow more room for users to express themselves in a way that goes beyond functional achievements, possibly that might have something to do with the smoothness and very fine nuances the instrument affords the player. But the distinction between analog and digital interaction, from the user point of view, has become very blurred by now; first with the advent of graphical user interfaces (GUI) and pointing devices, then with new tracking, sensing, and presentation techniques partly deriving from virtual- reality technology, and with tangible user interfaces (TUI) where physical objects are used in analog mode to interact with the digital artifact, and lately with the breakthrough of gestural interaction. The stage is now set for new applications and forms of interactions that could be much more like violin playing. In fact, we already have some artifacts and systems like these around, for instance, in the form of games that with quite simple interfaces invite and compel the user to develop sophisticated interaction skills, often virtual versions of existing “real” games and sports. Using a gesture-controlled golf simulator seems to be close enough to the “real” thing to let expert golfers show their virtuosity in the simulator.

Although we have focused on the violin as a possible model and inspiration for future artifacts and systems inviting virtuosity, we do not want to exclude designs with high external complexity from consideration in this respect. After all, the piano and other keyboard instruments, externally much more complex than the violin, also often figure in virtuosic compositions and performances. Still, the external simplicity of the violin by its very sharp contrast to older-style digital artifacts and systems perhaps makes it a more intriguing and challenging model for a designer. We believe that virtuosity as a manner of interaction could, and maybe even should, be reconsidered and revisited in interaction design. There may be situations and technology use that would benefit from such a perspective. The history of virtuosity is rich, ranges over many fields, and might provide us with new insights into interaction design possibilities with contemporary technology.

There are of course arguments against a move toward virtuosity in the field of interaction design. One obstacle to virtuosic interaction with many digital artifacts is their short market lives and rapid development: the violin has been around for hundreds of years with hardly noticeable changes, giving the community of users ample opportunities to develop a culture of virtuosic use. Many of the new digital artifacts, in contrast, are on the market less than five years before they disappear or are superseded by completely different designs, sometimes building on new and different technologies. Another argument against virtuosity is of course that it requires extensive training over a long time period. We admire virtuosity since most of us do not have the time and maybe talent to engage in training to the degree needed. So, to require virtuosity from users may restrict the group of potential users to almost none.

A more obvious factor working against virtuosity is that while it might be a sometimes-desired manner of interaction, such extreme interaction is not typically what designers and users look for. In much everyday artifact use, most users care for little more than very basic performance—and there often is a conflict between the requirements of high-level performers and low-level performers that prefer low interaction complexity. But, maybe a general change of attitude and the demands of professional users can change that; we have actually come a long way from the time when “user- friendliness” was the key criteria, and we still keep moving."

Maeda, J. 2006. The Laws of Simplicity. Cambridge, MA: MIT Press.
Mollerup, P. 2015. Simplicity—A Matter of Design. Amsterdam: BIS Publishers.

Wednesday, March 21, 2018

VR, authenticity and killer apps

The hype around VR has lately been growing and ads are making the case that it is time to take on this new form of interaction for all kinds of applications. However, in a great article (published by TechCrunc) the writer  Sibjeet Mahapatra argues that there is a major problem in the world of VR. The problem is the missing killer app.

Mahapatra discusses what he sees as the two values that VR can offer, primarily a sense of presence. But he also argues that VR still lacks when it comes to authenticity. The author makes a good case for what is needed for VR to really become something wanted by a larger audience. So, a good read about interaction and interactivity.

[And of course, I always like someone who  mentions Rorty and his "experience machine"!]

Monday, March 19, 2018

PhD course on "The elements of interaction"

I am just back from a trip to Europe where I among other things taught a two day Ph.D. course called "The Elements of Interaction". The course was organized at the department of computer science at Aalborg University by my colleague Peter Axel Nielsen.

It was an intense experience. Two full days completely focused on our new book "Things that keep us busy--the elements of interaction". We worked through almost all chapters in the book. It led to wonderful discussions. The doctoral students were great. They were curious, critical and inquisitive. And to me, it was a great way of exploring if the content of the book make sense and work for others than me and my co-author Lars-Erik.

Based on the experience, I learned two things. Our book seems to work fine with PhD students and they were able to relate the content to their own research in ways that might help them. Secondly, to teach a PhD course in this format, two full days, is excellent. It leads to complete focus. I will definitely argue for this format when I have the chance.

Thursday, March 08, 2018

When is a copy and the original the same

In an interesting article, Byung-Chul Han examines the notion of what is an original artifact versus a copy. He explains the different notions in the East and West in a way that is relevant to anyone thinking about design and creativity. The major argument that Han makes is that in China (and other Eastern societies) the notion of what is an original might appear as strange to us in the West. According to him, in these cultures, a perfect copy is the same as the original and has no greater value than the original. The article tells a number of fascinating stories of when this difference in thinking between East and West has led to serious misunderstandings and conflicts.

I was intrigued by this article. I have no idea how correct it is and how true it depicts the cultural differences, but even if it is not a true depiction, it does raise a lot of exciting questions about how to think about what is an original and if an original should have any particular status. Again, all relevant questions to any designer.

I have earlier commented on two other books by the philosopher Byung-Chul Han on this blog. See links below.
"In the swarm"

"The burnout society"

Btw, the online magazine Aeon, where this article published, is excellent!

Monday, March 05, 2018

Interaction and Complexity

One aspect of interaction that keeps emerging is related to complexity. A lot of people complain that interacting with systems and devices today is too complex. As a natural reaction to that, a lot of designers argue for simplicity as an important design principle. But what is complexity when it comes to interaction and why does it appear? In our recent book "Things that keep us busy -- the elements of interaction" we spend two chapters on interaction complexity and the related notion of control.

We do this by examining what interaction complexity is and what causes it. This leads to a theory (or model) of interaction complexity that consists of four different types of complexity. This is what we write (on p 85).

"We will identify and define four main loci of complexity of an artifact or system (see figure 5.1), all with respect to its designed purpose:
      1. internal complexity
      2. external complexity
      3. interaction complexity
      4. mediated complexity
These four loci should not be thought of as different measures or types
of complexity; they represent a rough division into the main (more or less abstract) locations where complexity is residing in varying degrees, and manifesting itself in various ways."

and figure 5.1 lays out how these different forms of complexity relate to each other.

After having worked with this model for quite some time, I find it quite useful and it helps to understand many aspects of interactivity and its relation to complexity. One of the major consequences of the model is that it indicates (strongly) that there is no easy "fix". To design for simplicity does not have any optimal solutions, every design decision about how to handle (or where to put complexity) leads to serious trade-offs that are inevitable.

This is why I believe that understanding this model can help and prepare every interaction designer to better approach the design of any interactive system and device.

Monday, February 26, 2018

Three new books on my desk

Just received three new books. Looking forward reading them. Especially since I know the authors of two of the books.

Friday, February 16, 2018

Nope, there is no 'hack' or 'fix' for designing

It seems as if we live in a time where everything can be solved with a 'hack' or a 'fix'. There are infinite websites, blogs, Youtube videos that show how to 'hack' a specific problem or even your life, 'life hacks'. The idea is that there are some smart, maybe almost 'magical', ways of doing things that some people know and now, finally, they are sharing it with the rest of us. You can find videos that say "You have been folding your laundry the wrong way" or "How to peel a banana the right way". These examples are of course harmless but it seems as if we are seeing a shift in mindset. People want quick fixes, smart hacks, that won't require long periods of learning, practice and experience.

It is possible that this mindset is also seeping into areas where it is not appropriate. For instance, the process of designing, of becoming a designer, is not something you can do or become by learning a few tricks or hacks. In the midst of the ongoing growth of design and design thinking (which is mainly a good thing), there is a misconception that you can spend a few hours in a workshop or maybe just read about some design "hacks" and you will become a designer. To me, this is a serious problem that leads to unhappiness and backlash. First of all, people become unhappy when they realize that their design efforts don't work or lead to good designs. Secondly, design, as an approach to change, will be seen as not working and we see a backlash.

I have never seen anyone propose that all you need to do to become a proficient scientist, musician or artist is a 3-hour workshop. Why is designing and design thinking seen as something that anyone can acquire without almost any effort? What does that say about how designing is understood?

Friday, February 09, 2018

"The Design Way" in Spanish

Fondo de Cultura Económica in Mexico City is publishing the Spanish language edition of our book "The Design Way -- intentional change in an unpredictable world" (MIT Press) by Harold G. Nelson and me. This is quite exciting and it will make the book accessible to an even larger audience.

We are also quite happy that the book has now been referenced almost 1.000 times by other authors. I hope that means that at least some of them have read it, or at least parts of it :-)

We do not have any date for when the Spanish version will be out, it will probably be a while.

Thursday, February 08, 2018

Creativity unveiled

My friend and colleague Harold Nelson pointed me towards a wonderful book "The Creative Architect -- inside the great midcentury personality study" by Pierluigi Serraino.

This book presents in a beautiful way the enormous volume of research done at IPAR (Institutes of Personality Assessment, today IPSR at Berkeley) during the 1950's and 60's led by Dr. Don MacKinnon. The purpose of the research was to create a deeper understanding of creativity.

In this new book, Serraino presents the background to the studies, how they were conducted, who was involved, and the final outcomes. The core subjects of the studies were some of the most famous and influential architects in the world at the time.

It is fascinating to read about the work that MacKinnon and his large team performed and the incredibly ambitious research approach they used. They performed studies that were heavily data-oriented, quantitative and analytical, based on highly detailed and personal reporting and observations of the subjects.

The chapter of the book called "Creativity unveiled" is absolutely amazing in its profound understanding of creativity. I could not stop underlining paragraph after paragraph of insights that the research had led to. Insights that in almost every detail resonate with my own understanding of creativity and design. Everybody interested in design and creativity should read this chapter!

This book does not only present a wonderful understanding of creativity (and design), but it also shows how fast knowledge is forgotten. There are no studies of this magnitude today. The size of the study, the ambitious methods, the detailed analysis is impressive and inspiring. Unfortunately, most of these results are not used today or even referenced even though they fit extraordinary well with what a lot of research today about creativity and design.

Wednesday, February 07, 2018

Design Instability - notes on design complexity

For quite some time I have had the pleasure to work with Zeljko Obrenovic on the topic of design complexity. We recently self-published a book on the topic, called "Design Instability - notes on design complexity". It is a short book that explores the complexity of design and why design is not easily simplified.

We describe the book like this:

"Designers deal with an overwhelmingly rich reality, undetermined requirements, highly intricate
social situations, lack of information, as well as lack of resource and time restrictions. Every designer is faced and challenged by this complexity. Discussing complexity in design, however, is difficult, as complexity is a very loaded and often vaguely defined term.

With this book, we want to better describe and give a structured definition of what design complexity may mean. We believe that such more structured overview can help researchers and practitioners to better understand some of the experiences designers are going through while designing. We also believe that in the education of designers it is useful to picture design complexity as an essential part of design that has to be accepted and dealt with, and not as a problem to avoid. We believe that a more structured understanding of design complexity can make such education possible.

A central thesis explored in this book is that complexity characteristic for design activities emerges from an inherent *instability of design activities*. We will argue that this instability is a consequence of messy, dynamic, highly interdependent and unpredictable dynamics between design situations, design outcomes, and design resources.

Design situations, design outcomes and design resources are complex structures that are not independent of each other. They are also continuously changing, in great part due to forces that are beyond designers' control."

You can find more information here:

Tuesday, February 06, 2018

Interface Thought Styles -- how to think about and understand what an interface is and can be

With the development of sensors and conversational interactions, it is obvious that what an interface is, becomes less clear. This is something that we address in our book "Things That Keep Us Busy - the elements of interaction" (MIT Press, 2017) when we develop the concept of "faceless interaction".

However, what an interface is or can be understood, has a history. In the book, we devote a chapter to what we call "interface thought styles". (The idea of 'thought styles ' is from Ludwick Fleck and simply means a particular way of thinking about a phenomenon. A way of thinking that influences and in some ways determine what can be thought and known about the phenomena. Thomas Kuhn developed his idea about paradigms heavily influenced by Fleck's ideas.)

So, what are the different ways we can think about the interface? Well, in Chapter 2 we write:

"The notion of interface has developed over a period of several decades and has been influenced by evolving technology and application areas. We have chosen to categorize some earlier and existing ways of thinking about the interface as belonging to four different thought styles: (1) surface- of-contact thought style; (2) boundary thought style; (3) control thought style; and (4) expressive-impressive thought style.

These thought styles are not distinctly related to a particular time period, particular technology, or type of design, but they have evolved over time and can be seen as stemming from different traditions. Fleck writes, “every thought style contains vestiges of the historical, evolutionary development of various elements from another style” (Fleck 1979, 100). Today, we can see that all four thought styles continue to be present and influential in our field, sometimes competing, sometimes cooperating with each other. The different thought styles are devoted to different aspects of the interface." (page 18)

We continue to define the four thought styles:

"The four thought styles explore the interface as
1. a surface of contact between matching objects (from the tradition of industrial machine making);
2. a boundary of an independent (self-sustained) object (from biology and traditional artifact design);
3. a means for controlling (operating, checking, steering) an object (from the design of complex machines);
4. a means for expressions and impressions, a target of interpretations and affectations (from human communication, architecture, and art)."

We do add a fifth potential thought style to this list later in the book when we introduce the field thought style.

There is not room here to discuss these thought styles in any detail (so, read the book :-)

It is quite surprising though, that the notion of the interface has not received more attention from the field of HCI. Of course, the interface has always been at the very core of the field. It is that "thing" through which all interaction takes place. But for something so fundamental, it almost seems taken for granted and not in need of any more sophisticated analysis or examination. Our position when writing the book has been the opposite. The very mundane concept of interface is crucial and definitely worthy our investigative efforts. And we would welcome others to join us in this endeavor.

Monday, January 22, 2018

HCI research and its neglect of complexity and systems

For many years I have wanted to engage in research that is focused on the HCI aspects of large complex interactive systems. But, I have not really done that. Almost everyday I hear stories from family members and friends about their experiences with their office, company and industry software. These are people who work with interactive systems in healthcare, insurance companies, retail, etc. they usually describe systems that have the kind of issues that in contemporary HCI textbooks seem to belong in earlier decades. The field of HCI is almost fully devoted to the kind of interaction that goes on in our private lives and very little in our professional lives.

I live myself in this situation. As an employee in a university, I have to use a number of large and complex systems that, from an HCI perspective, are extraordinary badly designed. As a field, we can, of course, blame the organizations and people who have the responsibility for these types of systems. And we tell them that they should work more in line with modern interaction design. However, even if that would help, it is probably not enough.

HCI research has a responsibility to also approach the dynamic and complex issues that are the consequence of large systems. Some of these issues are organizational and structural, some are related to the complexity of enormous masses of data and information, some are related to efficiency and effectiveness. In many cases, these are issues that are not addressed in contemporary HCI research with its narrow focus on 'user experience', the new and the cool, the interactive life of individuals, etc.

I might be wrong about this, but this 'blind spot' and neglect makes it difficult to argue for a broader societal impact of our research. As long as we only move forward, away from existing large problems, by focusing on the 'next', we are also escaping many of the inescapable aspects of designing complex system in complex environments in a way that would lead to people satisfied with their systems and work support.

Thursday, January 18, 2018

Excellent article about the BS in Higher Education

Well, now and then you stumble over a text that describes your understanding of the world in a way that both makes you happy to realize that what you experience is possible to express but also makes you quite sad when you realize how bad things are.

I have been in academia most of my life and I truly love it. It is a wonderful world of exploration, learning and challenges. Since I was a little kid I wanted to become a professor, now I am one and has been it for a long time. The world I love is however not working the way it should or could.

Christian Smith (professor of sociology at Notre Dame) has written a great article in the Chronicle of Higher Education called "Higher Education is drowning in BS". Smith describes this world I love with all is deficiencies. The text resonates with my own reality.  [I hope you can access the text].

I am, as Smith also writes about himself, by working in the system, supporting the system. I have done well in this system. My career has been good. But almost every day, I (as someone who can to some extent influence the system), think about some of the aspects that Smith mentions in regard to my decisions and actions and I realize that my actions are probably contributing to the increase of BS. Pause for reflection....and then we go again....

Wednesday, January 17, 2018

New book (soon to be published): "Critical Theory and Interaction Design"

I am very honored and happy to have been involved in a new edited book soon to be published by the MIT Press. The book "Critical Theory and Interaction Design" is edited by Jeffrey Bardzell, Shaowen Bardzell and Mark Blythe. Each chapter consists of a classic text from critical theory with a commentary from a scholar in the field of HCI. 

I had personally the opportunity and pleasure of commenting on a chapter from Herbert Marcuse's book "One-Dimensional Man: Studies in the Ideology of Advanced Industrial Society". This is a book that I have read many times and that has influenced my thinking in many ways.

I believe this new book to be an invaluable resource for graduate students in the field of HCI but also broader, such as STS, philosophy of technology, sociology, and more.

Here is the presentation from MIT Press.

Critical Theory and Interaction Design


Why should interaction designers read critical theory? Critical theory is proving unexpectedly relevant to media and technology studies. The editors of this volume argue that reading critical theory—understood in the broadest sense, including but not limited to the Frankfurt School—can help designers do what they want to do; can teach wisdom itself; can provoke; and can introduce new ways of seeing. They illustrate their argument by presenting classic texts by thinkers in critical theory from Althusser to Žižek alongside essays in which leaders in interaction design and HCI describe the influence of the text on their work. For example, one contributor considers the relevance Umberto Eco’s “Openness, Information, Communication” to digital content; another reads Walter Benjamin’s “The Author as Producer” in terms of interface designers; and another reflects on the implications of Judith Butler’s Gender Trouble for interaction design. The editors offer a substantive introduction that traces the various strands of critical theory.

Taken together, the essays show how critical theory and interaction design can inform each other, and how interaction design, drawing on critical theory, might contribute to our deepest needs for connection, competency, self-esteem, and wellbeing.

ContributorsJeffrey Bardzell, Shaowen Bardzell, Olav W. Bertelsen, Alan F. Blackwell, Mark Blythe, Kirsten Boehner, John Bowers, Gilbert Cockton, Carl DiSalvo, Paul Dourish, Melanie Feinberg, Beki Grinter, Hrönn Brynjarsdóttir Holmer, Jofish Kaye, Ann Light, John McCarthy, Søren Bro Pold, Phoebe Sengers, Erik Stolterman, Kaiton Williams., Peter Wright

Classic textsLouis Althusser, Aristotle, Roland Barthes, Seyla Benhabib, Walter Benjamin, Judith Butler, Arthur Danto, Terry Eagleton, Umberto Eco, Michel Foucault, Wolfgang Iser, Alan Kaprow, Søren Kierkegaard, Bruno Latour, Herbert Marcuse, Edward Said, James C. Scott, Slavoj Žižek

Tuesday, January 09, 2018

Why reason matter

I was randomly looking through my books today and the book "In Praise of Reason" by Michael P. Lynch was suddenly in my hands. I started to read it and realized that I had made a lot of underlining and comments in the book and remembered that I did write a book note about it. Looking at it now, it is clear that the message of the book is even more relevant and important today than in 2012 when I read it the first time. So, here are my notes from then.

Book note: "In Praise of Reason" by Michael P. Lynch

One of the most mundane activities that humans engage in is reasoning. We do it all the time. We try to find reasons for our own actions and for others (strange) behavior. At the same time, reasoning can be seen as the most advanced activity that humans engage in.

Reasons are the intellectual tools we use to convince others about our own perspective or solution. According to Michael P. Lynch, our society is facing a serious problem related to this daily human activity of reasoning. He argues that we have entered an era when many individuals and large groups do not accept the reasons of others as valid. There is a decrease in the trust of what he sees as the "common currency of reason", that is, there is less acceptance of the idea that we all, despite opinions and beliefs, are using the same fundamental set of rules and principles upon which we can constructively reason around a particular topic in a productive way. Instead, he argues that we see more people and groups expressing the idea that reasons are just a matter of belief. This leads to a situation where people do not have to listen to each others reason, not have to reflects upon the strength of their arguments, etc. Instead, people take the position that they are just wrong or even stupid. Lynch shows how this has become common even in parts of our society that are supposed to rest on reasoning and the exchange of ideas, such as in politics.

Lynch book gives a thorough and detailed account of the existence of objective reasoning that we all have to relate to and "obey". Even though Lynch is a professional philosopher and the topic is advanced, he manages to make his case understandable and exciting. To me, his argumentation seems both solid and convincing. His evidence for the existence of reason as something that is possible to see as common to all of us is both elaborate and elegant, but at the same time accessible. His description of the problems that will arise if we do not accept a common understanding of reason is straightforward and should give us all reason to fear the future.

Lynch also writes about something that I find extra interesting and that is a clear definition of science. He makes the case that a common understanding of reason can and should be based on an abstracted version of what constitutes the scientific approach. He writes "part of what makes scientific practice distinctive is that it is comparatively intersubjective, transparent, repeatable, natural, and adaptable." (p 93). These features give science the core quality that Lynch argues for which is an "open character". His detailed discussion about these qualities of science is highly interesting and is also relevant in a discussion about the difference between science and design.

At the end of the book, Lynch discusses the notion of truth especially in relation to Richard Rorty's idea of truth. Very interesting for those who are familiar with Rorty. He ends with a plea. He asks our society to seriously consider reason as a precondition for an open and democratic society. He argues that it is not just possible to develop a common ground and understanding about reason--it is necessary. Otherwise, our society will slide further down into a state when reason is not respected and other forms of convincing become tools, such as, money, power, violence.

I highly recommend this book. The points I mentioned above are just some from Lynch rich text. Read and think.

Thursday, January 04, 2018

Interesting critique of the hype of AI and robotics

I really enjoyed reading an article by Rodney Brooks titled "The Seven Deadly Sins of AI Predictions". It is refreshing to read a critical comment about the hype around AI and robotics. Brooks makes a serious attempt to argue against those who believe that AI and robotics will in a near future transform our society. Brooks argues that it will not. I agree with his general view of the slowness of technology development and even slower deployment into people's everyday lives. Read and reflect.

Wednesday, January 03, 2018

Digital technology and the disconnect with reality (Birkerts and Borgmann)

One of the books that have influenced me the most over the years is "The Gutenberg Elegies: The Fate of Reading in an Electronic Age" (1994) by Sven Birkerts. I read it the first time when it was published and I have returned to it regularly since then. Birkerts takes on what he sees as a fundamental shift in the history of humans. The shift is caused by the introduction and spread of digital technology. His eloquent arguments and examples lead to disturbing questions and reflections concerning the role and impact of technology. I have always felt that Birkerts is right in his observations.

In 2015, Birkerts published a new book filled with essays on the same topic and theme as in "The Gutenberg Elegies". I did buy the book when it came out but have not really read it until now. The title is "Changing the Subject: Art and attention in the internet age".

In this newer book, Birkerts returns to some of the fundamental issues he identifies with the way digital technology is influencing our society. Even though the overall idea of the book is similar, I find these essays to approach the issue in a more precise way. This does not mean that the arguments are clearer or stronger, actually, Birkerts was more forthright and direct in the earlier book, instead, here he takes a more poetic and delicate approach. It seems as if he is trying to capture what it means to live in a world that is not only 'using' a new technology but that is fundamentally shaped by it in a way that is mostly invisible and subtle.  He returns to what it 'feels' like to live today, what the 'system' does to you. For instance, he writes about e-mail:

"For one thing, to send and receive e-mail is also to move into the system of e-mail, to become implicated in the network. As a user I get both the frictionless burst of the contact -- the immediate breaching of the space-time divide-- and also the sensation, slippery but real, of taking a half step back from myself." (p 13).

He continues with a statement that speaks to me personally as true.

"This is one of the features of being inside the network mesh: incessant peripherality, and awareness of the larger world at every moment a click away. And because of this I occupy a different gravity field: I'm lighter, more porous." To me, this position resonates strongly with what Albert Borgmann is arguing with his 'device paradigm' and the loss of what he calls 'focal practices'. Focal practices keep us grounded, they connect us to place, time and community. Digital technology is extraordinary in its ability to disconnect us with place, time and community. It is exactly this ability that makes digital technology successful. Every time someone says "there is an app for that", it usually means that you can do something without having to connect to place, time and community.Both Birkerts and Borgmann argue that technology disconnects us from something more real, something more fundamental. I think that a lot of people on some intuitive level may agree with that argument, even though it is not clear what it means or what could or should be done about it.

It is possible to argue that the book, with all its essays, revolves around this idea that digital technology has "interposed a finely woven scrim of signals and distractions between me and my physically immediate reality". One of the exciting and scary aspects of Birkerts' ideas (and Borgmann's) is that the shift he is trying to capture and describe is not only invisible, it is slow moving, and has fundamental consequences. But we also have to remember that it brings new and wonderful powers to us all. What price do we pay for it? Who can resist it? Should we? Can we?

Featured Post

Why Design Thinking is Not Enough

If you go to Youtube and look for "design thinking" you will find a large number of videos with TED talks and other talks all expl...