Thursday, May 22, 2014

A growing problem in HCI research

After being to CHI and another gathering of HCI related people in the last few weeks, I have to share one reflection. I am not sure that my observation and definitely not my interpretation is correct or not, so comments are welcome.

The observation is that the field of HCI research is growing in size but also in scope. What today is considered to be HCI research spans a far wider area than every before. HCI today includes research that traditionally might have been seen as engineering (for instance, design and development of devices and systems with the purpose to explore technical possibilities), it also includes research that traditionally would be seen as within the social sciences or behavioral sciences (the latter has of course always have been part of HCI), but also research that can be considered humanistic or cultural studies, political or global studies. The research has also expanded in scope when it comes to application areas such as education, health care, transportation, entertainment, sustainability, etc.

This expansion of the field is of course exciting and has led to new perspectives and new knowledge that has enriched HCI. However, at the same time I am concerned by this development. It is possible to see this development as an expansion that leaves an empty space in the middle.

I would argue that a large part of todays HCI research could and maybe should be seen as research in other disciplines. For instance, research related to education should be evaluated and published in educational research venues and contribute to that field unless there is also a serious contribution to the core of HCI. This of course raises the question of what is the core of HCI.

If research in HCI do not in any sense contribute to our understand of human computer interaction in some general or universal sense, and if it is only an application of what we already know in yet another field, then it may be a contribution to that application field but not to HCI.  So, if someone applies HCI theory and knowledge (whatever that is) in another field to explore and examine a phenomena without bringing back some serious insights to HCI theory and knowledge then it is not HCI research.

What this type of expansion leads to is unfortunately in many cases research that do not contribute in a serious way to the core of HCI while also being questionable research in relation to what is the standard in the "other" field. If the research really contributed to those other fields then the research should be evaluated and published in those fields.

Ok, I understand that this argument raises a lot of issues especially around the notion of what is the core of HCI research but I see it as important for our field to discuss those issues if we want to be able to produce knowledge contributions that are distinct and valuable in relation to the contributions from other disciplines. HCI research is not going to be successful or recognized by how it is doing research or by how it is able to "use" knowledge from other fields, it will only be successful if there is a core knowledge contribution that is of a kind that no other field really cares about or produces knowledge about. HCI research will not become successful by expanding the field, not by approaching and including more application areas and topics. HCI research will only be successful if we can offer something valuable at the core that constitute well developed knowledge that no other discipline has done or will do, and that can be valued by its own merits.

8 comments:

Blair MacIntyre said...

I think this sort of thing (research whose contribution is primarily in another field being presented in CHI) can be a problem in all fields, but is only really a "problem" is a non-trivial amount of the work in the conference is of that sort (i.e., if too much of CHI is papers of this sort, such that it gets a reputation as a dumping ground for bad work in X, or if a new thread of CHI evolves that is most work in field X but also seen as not good).

But, it can also be the case that those other fields have a narrow definition of what constitutes good work, and by looking at it through a CHI lens, we get contributions that influence CHI and the world, even though the other field may not respect them.

In ISMAR (the AR conference), for example, there is often a worry that the computer vision work might not be acceptable in the main CV conferences (CVPR, ICCV, etc). However, ISMAR focuses on the challenges of building CV systems that really work, which those other conferences focus on more theoretical results; the long running joke is that to the CVPR community, "vision tracking" is a solved problem (since the theory is old). But making things work in real situations is hard.

Similarly, here, it may be that what passes for research education or psychology or cultural studies (or wherever) has a style of work or a standard of "contribution" that prevents us from exploring real systems in practice in the real world. We could do an education study or use education methods, for example, in a way that is too messy or small for the education community to care, but still gain insight.

The danger, of course, is that the methods and results are just badly done, not simply being used as best they can to address other aspects of a bigger problem. I think it is incumbent on the CHI community to ensure that when we get more than a paper or two that bridges to a certain area, that we hold researchers up to a standard of knowing, and explaining, how their methods and theories and work both fits and doesn't fit with those other communities, and why CHI is the right target.

I don't think that we need to have a "CHI" impact on some core "CHI kind of knowledge". Rather, we need to have impact on our understanding of computing and it's relation to humans and how they interact (with technology, each other and the world).

Erik Stolterman said...

HI Blair,
Thanks for the comments, very interesting. You make a good argument that I mainly agree with. I agree that any field can "use" knowledge and approaches from another field that transcends what has previously been done in both fields, I have no problem with that. So, I agree with your comment on the "non trivial amount".

And of course, now and then there are field cross-overs that lead to new fields. I very much agree with your sentence

" I think it is incumbent on the CHI community to ensure that when we get more than a paper or two that bridges to a certain area, that we hold researchers up to a standard of knowing, and explaining, how their methods and theories and work both fits and doesn't fit with those other communities, and why CHI is the right target."

And I think you in your last sentence give a simple but real definition of what constitutes the core of HCI :-)

Erik

logicalrealist said...
This comment has been removed by the author.
Sean said...

For the most part, I've been excited by and comfortable with HCI and the conferences I attend not having a narrow disciplinary scope.

This might be partly because I'm not convinced that HCI *is* a discipline or that it should have a core kind of research or flavor knowledge, so much as it is is an application area and community that benefits from the perspectives of many disciplines. (This is a research post, but as a tangent on the teaching side, I'm also skeptical of HCI undergraduate programs that don't have a strong disciplinary perspective.)

Discipline or not, I agree that scope issues are often tricky. Similar to Blair's comments, I hope that authors will clearly communicate in their papers why they are choosing to communicate work with whatever disciplinary perspectives to the HCI community, beyond simplistic design implications. I hope that review processes are able to evaluate whether work supports the claims it makes (which is made more difficult by a breadth of methods), and whether any given piece advances the conversation and work in the community to which it has been submitted for publication. I don't think that the answer is to articulate boundaries, though, but to continue to innovate with reviews processes and venues to help get the right work in front of the right audiences.

(sorry, having issues with comment posting)

Erik Stolterman said...

Hi Sean
Thanks for your comments. I appreciate them. I do believe that HCI is a discipline and not an application area. I agree with you that it is not an answer to articulate boundaries, instead I am arguing for articulating a core which is possible to have without boundaries. Then research can be close or farther away from the core and if it want to be seen as HCI research it has to make the argument that it contributes to the core in some way (in some cases even changing or moving the core of course)...
thanks again
erik

Jonathan said...

Erik,
I understand what you are saying but if HCID - still stands for Human Computer Interaction and Design then inclusivity (even in depth integration) of all things related to those areas is required. Even more so if we are to take a holistic view.

It is that whole 'H' or Human part of the moniker that is mucking things up. The CID parts could be more easily silo'd and cut up into a protected space and field. With the inclusion of Humans, however, we must (if we are authentic and honest) consider the system and relationships of US, as we are and our way of being in the world.

If the papers, areas of study and broad scope further that exploration, bust down silos and recognize the world in which the 'H' exist and acts then I am for it.

I can see some issues - those that already have carved out a silo in academia will of course cry foul and then there is the other issue of "how do we call it 'our field' when there is this muddled up bricolage of other fields?

This same issue extends outside of academia as well - simply look at any discussion on Linked in when the words UX, IX or Design thinking is mentioned... or look at job post and descriptions.

What you bring up about conferences and papers certainly contributes to the confusion and identity issues, at least if we use a lens of trying to carve out a space.

Perhaps a new lens is required, a new moniker which gets rid of the Humans in the equation or maybe we can own the idea of bricoleurship as the defining distinction of what we do.

Jeffrey Bardzell said...

I have posted a response to this on my blog, in case anyone is curious. http://interactionculture.wordpress.com/2014/05/26/hci-as-core-or-relation/

Vintage Leather Bags said...

Thanks for the knowledge very interesting article