Tuesday, October 27, 2015

Donald Schön – some reflections on his popularity

[This is a text a wrote many years ago for a workshop devoted to the work of Donald Schön. Is has been published in Swedish but I don't think in English. I apologize for the language in this text. This is also too long for a blog post but... A PDF version of this text can be downloaded here.]


Donald Schön – some reflections on his popularity
  • "Working methodically is the order of the day, and during this collective work in closed ranks it can happen that individuals forget to make use of their finest, most individual gifts. Their thoughts are for a time completely occupied with the task at hand, the one most available for being carried out according to plan. They become removed and lose feeling for that which is less tangible and only with difficulty perceivable, and as such for some questions they no longer have the necessary refinement." (Hans Larsson, 1892, Swedish philosopher, my translation)


The scientific approach creates endless data, information and knowledge and perhaps even insights. The truth of this can hardly be denied. In a society experienced by many as too complicated and changeable, such an increase in knowledge ought to create security and hope. Somehow it doesn’t seem to work that way.

At the same time that our knowledge grows, it appears to become more difficult to formulate knowledge and insights that are useful - at least when it comes to day-to-day professional situations. A lot of the knowledge produced by research isn’t experienced as being directly interesting by someone who has to act, has to make decisions, has to choose direction. Obviously, the truth and meaning of this development vary from area to area, but for anyone working with design the situation appears to be something of a paradox--the more knowledge we have, the more difficult it becomes to design.

The insight concerning the breadth and significance of design as a very special activity in our society has been gaining ground in the last decades. Increased system thinking has led to a deeper understanding of the enormous complexity surrounding every decision that gives our society form. We see more clearly relationships between economic systems, environmental aspects, questions of ethics, and esthetics. In modern design research, the limits of knowledge are constantly being challenged and expanded, and new aspects of design work are mapped out, conceptualized, and penetrated in detail.

While a comprehensive coherence seems to be ever more wished for but impossible, research shows that all aspects, each one in itself, can be developed endlessly deeper, and that together they create a complexity which is much greater than anything a single individual can comprehend. In many ways it has become impossible for a designer alone to take it all in and even more difficult to make use of the results of research in his or her daily work. In many ways what we’re experiencing is a knowledge paradox. The more knowledge we have, the more we feel that we don’t know enough to make a good design choice and decision.

Obviously a large portion of all research does not have as its intention the creation of knowledge that will support practical design work, nor should it. However it seems as if a large portion is directed toward helping and supporting practitioners. In my opinion, almost none of the design research being carried out today achieves this goal. In spite of the fact that researchers resolutely assert that the results have practical significance, this knowledge largely leaves practitioners indifferent. But there are exceptions - one such exception is the work pursued by Donald Schön.

Schön succeeded in formulating ideas and results in a way that is attractive and accessible for a broad range of readers. We can see his work referred to in a number of different disciplines. How can this be? What’s the difference between Schön’s research and other examples of design research? What makes Schön’s research so successful? Why is it so easy to get students both to understand and to attempt to use Schön’s ideas? Is it possible that Schön escapes the paradox discussed above?

There are obviously a number of possible answers to these questions. It’s possible that Schön’s ideas are in some way correct or true and that gives the results great penetrating power. But it’s also possible that Schön has a way of presenting his results that attracts readers because of its strong rhetorical force or appealing language. It could also be that Schön’s fundamental philosophical and epistemological approach led to results that can’t otherwise be reached. Or it can be that Schön’s personally formed research method, built on a qualitative and phenomenological tradition with roots in a pragmatic view of knowledge, led to results which are easy to understand and take in.

In this context I take the freedom to approach the question in a more personal way. The reason for this is not that I’m less interested in describing Schön’s work based on the recognized theories and categories of research methodology and conventional science. Rather it’s because I doubt that the answer to Schön’s popularity and significance can be found through such an analysis. So, at the risk of being altogether too simple and commonplace, I want to show what it is that I see as the most characteristic traits in Schön’s research and what also makes him both popular and significant.

First, I want to briefly comment on some of the other previously mentioned possible answers. To begin with it’s possible that the reader is attracted by his texts because they’re well written and they offer an inspirational reading. That language and literary presentation are significant even in scientific texts is not unknown. Many of our greatest thinkers have also been masters of style. It may even be difficult to find meaningful thinkers who don’t control the art of literary presentation in a conscious and stylistically sensitive way. Consciousness of that sort has occasionally taken the form of texts that are neither easily read nor easily understood. Kant and Heidegger are two examples. But even there, language and rendering are a very important part of the work being presented. It isn’t by chance that the language has the form it does.

Another answer could be that Schön’s rhetoric and artful argumentation are so well developed that the contents of his work does not matter. It’s evident that any text intended to reach a larger public must have the power to convince both in terms of presentation and argumentation. Schön fulfills both of these demands. But there are other design researchers who fulfill these demands without it meaning that they achieve the same success. We can never exclude the possibility that as readers we are dazzled by an argumentation that appears to be reliable without necessarily being based on more profound material. And in general we like to be dazzled.

Another interpretation could be that his texts simply give the reader what the reader wants. The texts may just confirm and support common sense interpretations of design work that everyone could make. The texts perhaps strengthen general prejudices without critically disturbing our habitual patterns of thought. One sign that could confirm this is one of the commonest reactions that I get from students who have just read Schön for the first time. It’s not unusual for them to say that the text was easy to read and that the contents were ”Okay, maybe nothing new but ok.... ”. However, it usually turns out that they have seldom understood the text in a deeper way, and above all haven’t realized the consequences of Schön’s explanations. In many cases, further reading and discussion leads to completely new evaluations of the texts.

The cause of the students’ reaction is to be found less in the superficiality of the text than in its deceptive character. Schön’s simple language and uncomplicated, straightforward argumentation are easily interpreted as a sign that the contents are also simple and straightforward. That lead, in the case of the students, to the conclusion that what Schön does is only to give a very superficial but true picture of design work. It’s my assertion therefore that Schön to a certain degree is not successful because of his language and presentation skills but actually suffers as a consequence of his language and his rhetoric. That leads me to believe that it isn’t in the language and presentation that we find the key to Schön’s success. So, what can it be?

Comprehending

We’re all familiar with the possibility of dividing knowledge into different forms with different objectives, such as understanding, comprehending, explaining, predicting, controlling, mastering, and prescribing. In design research, the dominant view seems to be that knowledge should be useful primarily for predicting and accordingly for prescribing. I’m not here going to engage in the growing criticism concerning this view during recent years. It’s enough to say that the criticism has largely been directed at an overly simplified view of the potential for controlling the results of the design process through methods and detailed planning.

When Schön takes on his research, he seems to be driven by an irrepressible and genuine interest in understanding and comprehending. He seems to be constantly astounded by the reality he studies—which is primarily the practice of the professional (designer). His astonishment is filled with fascination and even admiration. Reality isn’t to Schön, as for other design researchers, chaos of irrational actions that need to be corrected. Where other researchers see a reality that needs to be ordered and directed, Schön sees a rich and complex reality, full of life and people who try to do their best. For him it is a reality that above all should be something we should try to understand. Only by having a deep understanding of practice will there be a possibility of changing it. Where others see orderly knowledge as the answer to the problems of chaotic practice, Schön sees the problem in a blind faith in orderly knowledge.

Such a point of departure definitely creates the conditions for a form of research that can reach practitioners, in as much as it accepts reality as both existing and reasonable, and practice as both rational and comprehensible. It is research that becomes very practical even if it is driven by the urge to understand and not to control.

Firmly rooted

One aspect of Schön’s work is that he seems to be so stable. To be firmly rooted means that one knows where one is standing and looking, which role one has as an observer. It also means that the status of what one is looking at is known. When you are firmly rooted, there is all the time in the world to devote oneself to this ”looking” - it creates a peacefulness and a stability which is expressed in interpretations and texts. There is no stress to constantly seek new points of departure and new perspectives.

Schön never diverges from the task of trying to understand the expressions and conditions of practice. The point of departure is always the local activity, the weighing and decision-making of the practitioner. Schön never gets caught up indulging in empirical details, nor does he disappear in an abstract investigation of theories. As a reader, one feels that the objective is always close at hand and that it is both tangible and concrete.

But it isn’t just the steady objective that gives Schön consistency and stability. It is also a number of fundamental assumptions regarding the nature of reality, which constantly steer the work and like a rubber band pull the studies and the reader back to the central questions.

One such fundamental assumption is that the actions of professional practice are basically rational, that is, possible to comprehend as sensible deeds. That’s true even in situations where the results of the deeds don’t fulfill expectations. A lot of design research takes these failures as a point of departure and concludes that practice is irrational and to a large degree incorrectly carried out. And that what is needed is completely new methods and approaches that can transform practice. In such a perspective there is often no interest in thoroughly studying current practice since it has failed - and therefore nothing to have as a model for good practice.

For Schön, common practice is always interesting, even when it leads to bad results or seems irrational. He sees common practice as a manifestation of the dominant, as a consequence of practical conditions and restrictions, which must be more closely studied and understood. In this way, practice for Schön has status as the richest and perhaps the only source of true knowledge and understanding. Actually, all we need to do is to carefully study this reality without the preconception that it is basically irrational and in need of ”repair”. That means that what’s most important to study is our way of thinking about reality. We act and we reflect. And how we reflect is a result of the conceptions we have. For Schön, this can be demonstrated with the difference between seeing design as ”problem-solving” or as ”artistry”.

For Schön, these concepts can be used to illustrate the power of thinking over action. Seeing practice as ”problem-solving” leads practice to regard itself and its role in a specific way. The same is true of ”artistry”. There is a dialectical relationship between concept and action. What Schön shows in his research is that we can make our concepts visible and thereby also possible to change. In the same way that Schön employs this ”method” in his research, he also presents it as a solution for the practitioner. In the end there is only one way for both the researcher and the practitioner.

Capturing and stabilizing

There are many ways for a researcher to approach a rich and complex reality. Often it’s done by selecting one aspect to be studied. In order to be able to say something about this aspect, the researcher probably has to simplify reality and peel away complexity. The researcher has to stabilize and ‘clean’ his object of study. One of the most important tasks for a researcher is to be able to do this without losing relevant and significant information. It won’t do to reduce complexity to such a degree that the basis of its explanation also disappears.

The way Schön gets hold of reality is through capturing and stabilizing a study object. But he doesn’t do this in a way that makes everything simpler, that is, by reducing complexity. The object of study that Schön attempts to capture can perhaps most easily be called a designer’s skillfulness. That is to say, that which lets the practitioner achieves something despite the overwhelming and complex reality. The skillfulness of the designer can be seen as being a priori in Schön’s research. He lets this skillfulness function as a point of crystallization. All ideas and results bit by bit build up a richer picture around this core.

It isn’t an easy task that Schön undertakes. It’s difficult to empirically confirm the existence of something as diffuse and complex as skillfulness. It’s also possible to assert that skillfulness isn’t a general concept since it is something that is entirely individually based. And it’s also possible to assert that skillfulness - in the sense that Schön uses the concept - moves altogether too freely between the concrete and the abstract.

But it is just in this movement that we can see some of Schön’s strength. The ability to allow a central concept to develop while it is completely incorporated in the full complexity constituted by practice. Skillfulness becomes in this way something that hovers between the theoretical and the practical. In this way, skillfulness can be related to higly theoretical reasoning and concepts. But in the next moment be used to describe be concrete practice in a number of different case studies.

Getting closer

Another aspect of Schön’s way of working is his way of getting close to the phenomenon he has formed. It is commonly recognized that Schön’s examples and case studies are very effective. In his observations and descriptions Schön creates the ”Right, that’s the way it is”-feeling that in a convincing way affects the reader. The way Schön does this is difficult to classify.

With imminent risk of being altogether too categorical, one can see several typical and traditional ways of getting close to a phenomenon. One way of course, before beginning a study, is to develop a conceptual structure that can be used as a tool in the analysis and interpretation of a situation. This requires a well-developed theoretical construction. A familiar problem with this approach is that the construction becomes so governing that the observations in the study easily become the victims of the investment required by the theoretical preparations. As an opposite approach, which can be understood as ”grounded theory”, the empirical material is presented as being completely decisive. Getting close to a phenomenon unconditionally and without previously defined theoretical tools becomes the strategy. One wants to get as close to the phenomenon as possible and to create pictures, which are as rich and as deep as possible. The idea is that out of these pictures relevant and decisive analyses and interpretations will arise, and these in turn will be the foundation for further conceptual developments.

Between these two extremes there exist all the in-between forms, where we probably find most research. But Schön is difficult to put into one or the other of these two categories. In some rather unique way he succeeds in ”escaping” the demanding portions of both of these approaches. Schön doesn’t get close to his object of study with a well-developed conceptual structure or with highly specialized conceptualizing. Nor does he get close to his empirical material in a particularly methodical way, and above all not with the degree of detail and exactness that is usually required.

One can read Schön’s texts as though it’s neither the concepts nor the empirical material that prevail. Instead it is the well developed object of study. In order to achieve the greatest possible understanding for this object, Schön simultaneously approaches it from all sides. Concepts are captured and refined in interplay with both general and detailed descriptions of the empirical. We get a better feeling for the theoretical concepts at the same time that we get a better and deeper picture of the empirical. Through this process, what is being made more precise is the object of study. The Swedish philosopher Hans Larsson has in his book Intuition succeeded in capturing, in his wonderful language, how this takes place. He writes concerning the danger of making concepts too precise too early:

"It is an altogether too common piece of theater, that while we demonstrate with the cage of definition in hand, the bird that we think we have flies away, back to freedom." (Larsson, 1892, p. 60)

Larsson also describes how the development of an understanding in the way I attributed above to Schön must take place; how it is a delicate, slow and sensitive process.

"And every new concrete stroke he draws, makes the border somewhat more distinct for the concepts he wants to define; his thoughts hover less and less in the blue and get closer to the objective .... How many of these concrete strokes, from the pictures constantly appearing out of life, and out of the most deceptive, fleeting, inner life, are needed in order to achieve such a fixing." (Larsson, 1982, p. 65)

Larsson also presents a wonderful description of how a successful approach must take place. I can see only that the description corresponds well with many of the studies Schön has made.

"I’m quite aware: the borderline drawn up in this way is in a certain way wide, but the area is completely surrounded, and inside the drawn line is the object being defined. You want to tighten up the line, but you have to make sure that when you do so the object doesn’t disappear.
Don’t get too aggressive with your definitions. You want to get your hands on the truth: well and good - if you can! The secret of life is like a bird in the woods. Don’t become one of those who rush unskillfully around wanting to catch him dead or alive. Make your approach carefully and keep yourself still - so you may get to hear him sing." (Larsson, 1892, p. 67)

For some, getting close in this way may appear to be unstructured or too diffuse. The result may not achieve the desired stringent form. But I believe that just the opposite is true. Schön’s empirical material, that is, the case studies combined with the special treatment of concepts, are all performed according to Larsson’s advice. Schön knows how to ”make his approach carefully and keep himself still” and in that way, through his texts, we are able to hear the bird sing. In this way reality isn’t mutilated and overly simplified - but rather living and true. In this manner, the material is given a convincing power that feels not only possible but also true.

Getting close to a study object this way can also be described with the help of the concept notitia. Notitia requires approaching what one is studying with extreme exactitude. But it is not about getting close however you choose, not even with getting close with as much precision as possible. It’s a question of getting close to the qualities of what’s being studied. The psychologist James Hillman, a promoter of this approach, writes:

"Attention to the qualities of things resurrects the old idea of notitia as a primary activity of the soul. Notitia refers to that capacity to form true notions of things from attentive noticing. " (Hillman, 1989)

Notitia in this meaning is an instructive concept. Hillman wants to show that the most significant and deeply embedded characteristics that may be hidden in the things around us cannot be understood if we don’t abandon ourselves to ”attentive noticing”. It requires time, energy and precision to discover the deeper significance of things and actions. The art of practicing notitia in a way that leads to valuable results requires skillfulness. It is an art that must and can be trained and developed according to both Schön and Hillman. That’s what Hans Larsson describes in the introductory quotation of this text where he writes that we easily:
"…become removed from and lose feeling for that which is less tangible and only with difficulty perceivable, and as such for some questions they no longer have the necessary refinement." (Larsson, 1892)

Schön’s way of getting close to his object of study, his ability to say things about this empirical material, to make the complexity of this practice both visible and understandable, shows that he has this sensitivity - this ”necessary refinement”.

Moments and timing

An important part of what constitutes Schön’s work is his way of using his empirical material. In his books the empirical material doesn’t consist of large, quantitative investigations, nor does it consist of deep, qualitative studies. Schön doesn’t use surveys or interviews. What best describes his empirical material is to say that it deals with observations.

The observations that Schön mainly presents don’t cover a long period of time. In that way it’s difficult to even call them case studies - we could rather see them as brief time segments. Nor does Schön’s empirical material deal with detailed and precise notes and descriptions of what takes place in these brief segments. Rather it’s often a question of Schön having taken part in a single session or work shift. Or it can be a small experiment that he has carried out together with his own students. In his texts, Schön describes these situations in a simple and brief way. In some cases it becomes a longer description, but never according to rules or methodological guidelines from an accepted or developed empirical method. In any case, the way in which the work is carried out is not reported.

Despite his unorthodox empirical approach, Schön succeeds in making his cases both convincing and comprehensive. Also indicative is how Schön’s readers often seem to find it easy to recall his cases. Many readers can name and repeat their favorite among the cases described.

We could call this form of empirical research the study in the moment. In Schön’s texts it is difficult to see that the cases he chooses to describe are the result of a well thought out plan. Nor does he describe the whole case or situation. He often chooses to describe only a very short passage in a session. A passage which perhaps covers only a minute. Schön then lets this little time segment of practice be interpreted and analyzed - but again not according to any common methodological guidelines.

It’s easy to criticize Schön’s empirical work as being momentary, not representative, not consistent, and perhaps even careless. But why does his way of approaching empirical studies have such penetrative power?

Perhaps the most important explanation is timing. It is the ability to pick out a time segment that in itself carries something comprehensive, unique and characteristic. Such a well-chosen segment must have great integrity and clarity. When this is the case, an interpretation appears to be quite obvious and cannot be overthrown by alternative interpretations. Great integrity in the segments imparts believability and significance. To see and pick these segments is a skill that Schön mastered.

The long span

What is it as a whole that constitutes the strength in Schön’s work? Perhaps we can call it the capacity to bridge the long span. What Schön does is to create a balance point between the abstract and the practical, between the general and the specific, between detail and whole, between thought and action.

Through his slowly developed theoretical foundation and his well chosen empirical segments, Schön succeeds in bridging that span. Perhaps this bridging is something we all recognize more or less consciously - and maybe it is something we even long for.

Altogether too often we are confronted with research results that feel only abstract and theoretical or we are flooded by empirical material. Above all, we as readers are made to create the whole ourselves, out of the material we have to bridge the span, to create a balance point where all the threads, ideas and concepts gather and radiate together. Perhaps this task too often feels overwhelming. Perhaps Schön helps us here, and that is what attracts us.
But it’s not only that it attracts us by giving us a whole, but also that that which arises has a strongly convincing power. Schön’s results, despite all their faults, powerfully influence the reader and have also found readers in all circles.

It’s not difficult to direct a lot of criticism against Schön’s way of working. Almost any recognized scientific methodology gives a good foundation for such criticism. But the task for me in this text has not been to ask that question, but rather the opposite: why has Schön’s work won so much respect and why is it so appealing?

The aesthetic gestalt

So, what is it in Schön’s work that draws our attention? I believe that in the end it has to do with the fact that Schön himself was a designer. A book by Schön shouldn’t be understood as a complete presentation of a theory or as a presentation of empirically well-grounded scientific work. Rather his work must be understood as aesthetic gestalts, that is, as compositions. A composition that is well designed has integrity and an expressive character that greatly exceed every part or detail. A book by Schön has these characteristics.

As with every good design, we are struck by the whole that the design radiates. We see this gestalt first and become less dependent on single points, each one of which may perhaps have faults.

The gestalt that Schön presents for us thus has a strong aesthetic power. As readers, we feel this power and are drawn to it. One characteristic of such a gestalt is that, if it is well composed, it also has a strongly convincing power. Not the power to blind us or overwhelm us with its appearance or its availability. Rather it radiates coherence and integrity – and thereby a sense of truth.

It may be that we see Schön’s compositions as the result of his position regarding the realities of practice, and his struggle via ”authentic attention” and notitia to get close to this practice. It is because Schön gives us an interpretation of reality and its practice, which feels so true that we are attracted to it. In a pragmatic spirit we could also interpret this feeling of truth as connected to activity, that is, we feel not only that Schön’s gestalt is whole, but also that it creates the possibility for us to both understand and to act in a complex reality.

Schön’s results are used in many contexts as a foundation for design theory that connects practice with theory, specialization with generality, research about design with research for design. In most cases, we see general references to Schön, often to all of his research, but we see to a lesser degree references to specific results.

My interpretation of this is that Schön has created a foundation and a point of departure that many do not have. And he has done it in the form of an esthetic gestalt. It is a gestalt which feels whole and coherent and which has great integrity. It is a design - a fabric - with aesthetic qualities. In that his method and result are simultaneously both all inclusive and specific, there are points of entry to the fabric that make it possible for many to find their way in. In that way his work creates a gestalt around which other design research can take form, grow and develop.


Monday, October 26, 2015

What makes a research prototype/system a knowledge contribution?

Once again I have been involved in discussions on what is the role of a prototype or a demo system in HCI research. What is it that makes a prototype or a system valuable in itself and when do we know that it is a knowledge contribution? These are old questions in our field that for instance Jack Carroll and others discussed over a decade ago. However, I do not think we have a good answer to any of these questions.

I wrote a blog post on this earlier
http://transground.blogspot.com/2015/01/what-makes-prototype-novel.html
where I talked about a paper I wrote with Mikael Wiberg where we tried to answer or at least approach these questions. Unfortunately, I have not seen a lot of discussions lately and would like to see more people engage in this topic. It is crucial for our field when it comes to how we evaluate papers, demos and system at conferences, etc.

[If anyone has some good references, please send them to me.]

Tuesday, October 20, 2015

Paul Dourish comments on our article "Faceless Interaction"

I have earlier on this blog mentioned the article "Faceless Interaction" (see ref below). The article was recently published and we are both happy and honored that Paul Dourish has written a commentary on our article in the same journal (see ref below).

Dourish writes "In their article on faceless interaction, Janlert and Stolterman offer us an examination of a foundational concept in interaction design that is both generative in the conversations that it provokes and surprising for the fact that such examinations are not undertaken more often."

Dourish continues "Their examination is driven primarily by a need for conceptual foundations,
rather than an empirical effort to grapple with new interaction modalities, although
of course the rise of surface-based interactive devices, the increasing use of speech
and free-form gestural technologies, and the decentered interactional style of ubiquitous
computing all point toward a need to recognize the importance of what they
usefully term “faceless interaction.” The result is a genuinely novel approach that does
more than simply set the stage for a new model of interaction; it also provides a critical
perspective on the history of human–computer interaction (HCI).

Dourish ends his commentary with "I am delighted to see the sorts of deep conceptual examination that Janlert and Stolterman offer. I look forward to the conversation that it provokes."

You can find the whole commentary here: http://dx.doi.org/10.1080/07370024.2015.1022177

If you are interested in our article, just let me know: estolter@indiana.edu

----------------------------

Paul Dourish (2015) Commentary: Conduits and Communities, Human–
Computer Interaction, 30:6, 540-543, DOI: 10.1080/07370024.2015.1022177

Janlert, L-E., & Stolterman, E. (2015). Faceless Interaction - a conceptual examination of the notion of interface: past, present and future. In Human–Computer Interaction, Vol. 30, Iss. 6, 2015

Monday, October 19, 2015

Amazon Echo investigations

I have now had an Amazon Echo since May (see post) and it is amazing how many questions about the future of HCI the use of this device leads to, especially in relation to what Lars-Erik Janlert and I call "faceless interaction" (see earlier post). I am convinced that Echo is a type of device that in the next few years will fill our personal and private environments. Of course, it may be that devices such as Echo may disappear into the environment itself and be less distinguishable as a 'device' and become more a 'function' of the environment.

I have had the privilege to work for the last couple of month with a couple of excellent students who together with me are investigating Echo. We are trying to map what happens when people are confronted with 'faceless interaction' in the form of Echo. What do people do and think? How do they behave? And most interestingly, what questions about interaction in general does this lead to? We are starting to see some structure and some categories when it comes to the use of Echo, and it is already clear that some of them are not what we intuitively might expect, that is, Echo is not 'just another device', it brings something new.

We are right now planning some small studies and experiments and we hope to have our first paper ready later this semester. I see this research as exciting since it relates to some of the newest interactive devices so it is exciting from the technological and functional point of view, but at the same time I see it as really foundational since this type of device can help us to examine and reflect on aspects of the fundamental nature of interactivity and HCI.

Friday, October 02, 2015

Core and target systems: HCI research and the measure of success

HCI research is usually seen an academic activity meant to create knowledge about interactive experiences between humans and computers, and about the technology that makes those experiences possible and about the process of shaping these technologies. Design of any artifact and system is complex and has to satisfy many requirements, needs, wants, and desires, which means that it is not easy to know how to measure the overall success of an interactive system. This means that the measure of success of HCI research is equally complex.

I will not discuss this problem at any length here, only mention one aspect that I frequently see manifested in papers, articles and in phd dissertations in the field.

We might call the artifact or the system that a HCI researcher is developing, studying, or evaluating for the core system and the (social) system where the design will be implemented and situated as the target system (or context or environment). For instance, if an interactive artifact is supposed to support people to handle their emails better, the core system can be seen as the email application (device) itself and the target system is the way people handle and communicate via email in their everyday lives. If we measure the time people spend with their email applications it seems, at least based on anecdotal evidence, that we are moving towards a situation where email is consuming an increasingly large part of people's work day (at least in some professions).

It is commonly appreciated that email systems are not working well and need to be improved. This kind of observations about a specific core system in many cases drives research in a field and is commonly mentioned as the main argument behind a particular study or design of a new system in conference papers. The research ambition becomes to improve the core system. This usually leads to efforts to make email more efficient, faster, producing less email, make the email experience easier or maybe more pleasurable.

However, if we instead would measure and evaluate the impact that the email (core system) might have on the overall work load and effectiveness (the target system), we might get a different understanding of what the consequences are of email systems. For instance, it may be that the increase in time spent with emails actually increases the efficiency of the target system at a level that far exceeds the time that the core system requires. It may be that badly designed (according to certain principles) core systems lead to improvements in the target system that are desired.

We have all seen the typical research paper that describes a new idea implemented in an artifact. The idea is to improve certain aspects of a human activity. The artifact is built into a prototype and tested on students who are far from representative when it comes to the particular activity. The results show maybe that the 'users' liked the prototype but the research state that there are more research that needs to be done but the results so far are 'positive'. This is, however, far from a measure of success that tells us anything about the validity of the 'new idea' that the artifact was supposed to manifest. The 'new idea' is commonly argued for as a way to improve the target system while the evaluation is set up to only evaluate the core system.

For those who are well trained in systems thinking this is not new in any way. C. West Churchman wrote several books that in a wonderful way display the dangers when systems thinking is not involved in design.

The problem is of course that we are already today suffering in HCI research when it comes to anything that has to do with evaluation and testing of new artifacts and systems. The field knows that traditional lab tests only give partial answers and there has been a strong push and move to the 'wild', that is, to evaluate designs in the context where they are supposed to be used. However, the overall purpose seems still to be about the qualities and issues related to the core system and seldom about how the target system actually changes and is influenced. In some areas, this type of target oriented studies are called 'clinical studies'.

I am not arguing that all HCI research has to be measured by its real impact on target systems since I think that would lead to some form of paralysis in our field. But I think our field would benefit from a more in-depth discussion about how to deal with design oriented research (which is most all HCI research) when it comes to evaluations. Ok, this is a big topic and I have just scratched the surface here. A debate and discussion is welcomed.