Skip to content

******Interaction Gestalt And The Design Of Aesthetic Interactions [Lim, Y.]

February 9, 2010

Taken from: Interaction Gestalt And The Design Of Aesthetic Interactions
LIM, Y., STOLTERMAN, E., JUNG, H., AND DONALDSON, J. (2007) Interaction Gestalt And The Design Of Aesthetic Interactions. In Proceedings of the 2007 Conference on Designing Pleasurable Products and interfaces (Helsinki, Finland, August 22 – 25, 2007). ACM, New York, NY, pp239-254

Bibliographical Annotation

Lim builds upon the work of Petersen and Fiore attempting to align, or at least draw from, more qualitative design disciplines’ use of Aesthetics. Lim et al identifies a gap in practical knowledge in designing for the Aesthetics of Interaction. Attributes of design, in their argument, are a way to re-address this lack of practical knowledge by manipulating them to explore and shape the ‘interaction design space’. This manipulation is through a gestalt framework where the sum of the manipulations creates an interactive experience greater than the attributes (p239). This gestalt is experiential, as both Petersen and Fiore have argued based upon the Pragmatist philosophy of Dewey and Shusterman. Lim identifies that what is lacking in this ‘interaction design space’ is designers’ ability to explicitly shape interactions in an aesthetic way, rather than in an implicit way. HCI is striving for a quantified hermeneutical advance, for something that is qualitatively defined through personal experience and critique (pp239-240). Ideally Lim hopes that Interaction Gestalt can become a conceptual tool (p241), to create a framework upon which the internalized knowledge of aesthetics that graphic and interior designers work from (and to an extent product designers and architects,) can be appropriated into Interaction Design. The material used to create the designs in IxD is not the same they are distinctively different (p441 & p245).

Using Norman’s suggested three levels of perceiving beauty (visceral, behavioural, and reflective), Lim positions Interaction Gestalt as a framework to achieve the first, surface, visceral level of perception of aesthetics; whereas they position Petersen at the higher levels of behaviour and post-experience reflection (p244). In saying that, Lim still follows Dewey’s pragmatism as aesthetics as a holistic, whole experience (p234). Therefore, they argue, the attributes of Interaction Gestalt are contingent into being shaped by, and translated into the properties of the interactive prototype. Only then can users perceive and experience them. This relies upon a designer anticipating how an eventual user will experience the Gestalt. The designer then enters an iterative cycle to shape the gestalt (pp245-246). The attributes that form the interaction are descriptions of the shape of the interaction and not qualities of the experience (p249). This separation is akin to the Modernist design maxim of Form and Function. Two different parts, that when in relationship or flow, in a way creates a design Gestalt.

Where I begin to have reservations about this informative paper is, unfortunately, in its comparisons to the ‘visual’ design (graphic design by yet another term). The attributes of an interaction and the designer’s manipulation of them, is by Lim, equated to the manipulation by a [graphic] designer of a margin within page layout (p250). My problem is not one of fundamental disagreement but more that I currently perceive this as a tenuous link. I think I would have less of an issue if Lim et al had identified what the attributes of interaction are first. Then the comparison would be more transferable.

In my own research into repositioning VisCom into IxD, this paper is useful in presenting a possible theoretical and conceptual framework I can build upon. The term coined by Lim of “interaction design space” in which the attributes are manipulated is conceptually useful. Their positioning of the Interaction Gestalt only to Norman’s lower level of aesthetic appreciation is disappointing, as this would mean that VisCom influence could be restricted to the ‘visceral’ surface again.

Useful Quotes

Although there has been a drastic increase in the research of aesthet-ics of interaction, we still lack well-defined practical knowledge of how to design aesthetic interactions. In order to develop such knowledge, we adapt three important ways of thinking in designing interactions influenced by traditional design disciplines, namely, 1) understanding what it is that is designed—i.e. interaction, 2) knowing what is possible to be manipulated when designing interactions—i.e. attributes of interaction, and 3) mastering how to manipulate the attributes to shape the interactions.” (p239)

Although these directions of research are crucial to understand the role of aesthetics in HCI design, it is still not clear how practical and useful such approaches are in terms of designing aesthetic interactions.” (p239)

We argue that any interaction takes on a gestalt, a composition of qualities that “creates a unified concept, configuration or pattern which is greater than the sum of its parts”[A]. In this paper, we argue that this way of thinking about interaction as an interaction gestalt better invites designers to more concretely and explicitly explore the interaction design space to create aesthetic interactions, especially when comparing to current approaches that blur the relationships among user experience, interaction and an interactive artifact. In any interaction, the interaction gestalt is experienced by a user and evokes the user’s subjective experience of the quality of the interaction [B,C]. However, only thinking about the user experience cannot fully guide design-ers to explore a design space of possible aesthetic interactions in a concrete way. This means that designers should have knowledge of how to shape aesthetic interactions in a more visible, explicit, and designerly way. This is a kind of knowledge we are currently missing in HCI.” (pp239-240)
(A) Wiktionary, (25 February, 2007)
(B) Löwgren, J. and Stolterman, E.: Thoughtful Interaction Design: A Design Perspective on Information Technology. MIT Press, Cambridge, MA (2004)
(C) Svanæs, D.: Kinaesthetic Thinking: The Tacit Dimension of Interaction Design. Computers in Human Behavior. 13, 4 (1997) 443 – 463

The basic principle guiding our re-search is that we want to adopt and adapt certain ways of thinking through the use of a design language1, similar to what can be found in established design disciplines. This language includes: (1) a good sense of what it is that is designed—i.e. a design target—in our case the interaction itself which we call interaction gestalt, (2) a good sense of what is possible for a designer to manipulate when designing the design target—in our case, the attributes of the interaction gestalt, and (3) a good sense of how to manipulate these attributes in order to shape a specific design—the interaction gestalt.” (p240)

shaping the gestalt involves both imaging how the gestalt should be manifested in an interactive artifact as well as anticipating how users will experience the gestalt.” (p241)

When we discuss aesthetic aspects of interactions, we are not concerned with the aesthetics of everything. We focus solely on interaction, and how aesthetics can be understood in relation to that. To that end, our research is not about arguing the importance of aesthetics. What we are proposing is a conceptual tool that can practically guide and inspire designers in their design of interactions.” (p241)

When we look at traditional design disciplines—e.g. product design, visual design, interior design, and architecture, the knowledge similar to what we are proposing has been established, educated, and discussed for a long time. For example, visual designers know that they should be able to manipulate key attributes of visual products such as margins, shapes, typefaces, and spatial layouts in order to design the gestalt of the visual product. In a similar way, a skilled carpenter has intimate knowledge of his material and how to manipulate that material. However, the carpenter usually has no theoretical knowledge of use experience in any research manner. The carpenter trusts his or her deep internalized knowledge of what can be done and how it can be done with the material at hand in order to create something that is both beautiful and functional. The knowledge about materials for design has been a core part of training and education in traditional design fields [A]. We claim in our research that we need this kind of (internalized or internalizable) knowledge for interaction design as well. However, interactive artifacts are not the same kind of artifacts that are primarily tackled in traditional design disciplines. The dynamics, flexibility, and intelligence enabled by computing technologies embedded in interactive artifacts make the characteristics of such artifacts distinctive from other types of non-computing technology artifacts.” (p241)
(A) Ashby, M. and Johnson, K.: Materials and Design: The Art and Science of Material Selection in Product Design. Butterworth-Heinemann, Boston, MA (2002)

In the special section on aesthetics in HCI in Human-Computer Interaction (HCI) Journal, Volume 19(4) published in 2004 [A], Norman started the introduction to the section by describing the significant contrast between psychologists’ view and art historians’ view on aesthetics—beauty. The psychological perspective views and analyzes aesthetics through the objective and scientific account of how people perceive it—i.e. through which mechanisms people perceive beauty, or think that they perceive beauty. For example, Norman suggested three levels of perceiving beauty including visceral which is the most “surface” level that is tightly related to human sensory perceptions, behavioral which is the second level where people perceive aesthetics through operation- and action-based perceptions, and reflective which is the deepest level in which people go beyond their immediate perceptions of beauty and form their own meaning of beauty through intellectual judgment [B].” (p242)
(A) Norman, D. A.: Introduction to This Special Section on Beauty, Goodness, and Usability. Human-Computer Interaction. 19, 4 (2004), 311-318
(B) Norman, D. A.: Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, New York, NY (2005)

Although these studies have helped designers understand that there are different types (or levels) of users’ perceived qualities when they interact with products, and also have helped to see that these qualities relate to different aspects of artifacts, this knowledge does not yet directly provide usable knowledge for designers to creatively figure out how to embody aesthetic qualities into their design ideas.” (p243)

In this line of research, aesthetics is instead viewed as a holistic experiential outcome which cannot be separately treated as one type of component in addition to others. Aesthetics is equal to the most desired fulfilling whole experience. This view is much influenced by Dewey’s pragmatism [A]. The approaches dominated by psychological perspectives described in the previous sub-section do not deeply consider or discuss the holistic notion of aesthetics as a whole as experience.” (p243)
(A) Dewey, J.: Art as Experience. Perigee Trade, New York, NY (2005)

Petersen et al. [A] emphasize the importance of aesthetic experience as interdependency between mind and body experiences, and claim that the aesthetic interac-tion is not to be found in the artifacts but is what “emerges from the personal and interpersonal sensations, experiences, and reflections that is connected to the system […]. [A, p.271]” This perspective supports our perspective of interaction, which is that interaction is not something inherent only to the artifact but something that emerges through the inter-plays between people and artifacts. However, our proposal in this paper focuses more on the design of the immediate level of interactions and less on the high-level experiences that encompass lengthy socio-cultural contextual experiences over time. We also claim that these high-level experiences can only be formed by the collections and integrations of such immediate levels of interactions.” (p244)
(A) Petersen, M. G., Iversen, O. S., Krogh, P. G., and Ludvigsen, M.: Aesthetic interaction: a pragmatist’s aesthetics of interactive systems. In Proc. of DIS 04. ACM, New York. (2004) 269-276

To develop such insights about material properties is not easy, especially when it comes to interaction. The material we deal with is not tangible like plastic, metal, wood, or visual elements that constitute familiar building blocks in traditional design fields. The material we need to understand for interaction design is flexible, ungraspable, and phenomenal.” (p245)

it is essential to define and research what the shape of interaction is, which we call interaction gestalt, so that we can help designers articulate and manipulate this unusual type of phenomenon which does not have tangible shapes, and is flexible, ungraspable, and easily changeable [A].” (p245)
(A) Löwgren, J. and Stolterman, E.: Thoughtful Interaction Design: A Design Perspective on Information Technology. MIT Press, Cambridge, MA (2004)

The pragmatic perspective of aesthetics especially works well when we try to explain the aesthetics of interaction that does not have tangible properties because it emphasizes that the aesthetics is not intrinsic to the artifact itself, but to the way people experience it. As we emphasized before, interaction is basically viewed as a phenomenon that emerges in-between people and digital artifacts. It is not inside of the artifact. It is continuously going on and changing over time. Unlike physical and tangible gestalts, interaction gestalts are dynamic and difficult to grasp.” (p245)

The interaction gestalt also has its own types of attributes which can be used to shape, describe, and analyze the interaction gestalt. (…) The interaction gestalt is shaped by a set of interaction attributes that must be translated to and manifested in the interactive artifact properties in order to be communicated, perceived, and experienced by users. The interaction gestalt also has to be designed in a way that will evoke the desired user experiences. The designer has to anticipate how a certain gestalt will be experienced by a user, and that anticipation has to be translated back into ideas on how the gestalt should be shaped.” (p246)

Traditionally in HCI, interactions have been described by languages of 1) interface styles such as WIMP (widows, icons, menus, and pointing device), 2) forms of interface devices such as tangible interfaces and graphic user interfaces (GUIs), 3) actions that are supported by interfaces such as instructing, conversing, navigating, and browsing [A], and 4) object-based concepts such as spreadsheet applications designed following traditional ledger sheet forms [A]. Although all these approaches have helped conceptualizing and shaping interface designs, they have not directly supported the aspect of aesthetics when designing interactive artifacts. Major approaches for supporting aesthetics in interaction design have primarily focused on providing indirectly influential techniques such as storytelling [B]” (p246)
(A) Preece, J., Rogers, Y., and Sharp H.: Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, Inc. New York, NY (2002)
(B) Fiore, S., Wright, P., and Edwards, A.: A pragmatist aesthetics approach to the design of a technological artifact. In Proc. of the 2005 ACM Decennial Conference on Critical Computing: between Sense and Sensibility. ACM, New York. (2005) 129-132

The concept of interaction gestalt will lead us and hopefully other interested researchers in near future to contribute to develop an interaction design language as with, for example, visual designs—visual literacy [A]—which has been established, educated, and significantly contributed to shape aesthetics of visual designs.” (p247)
(A) Wilde, J. and Wilde, R.: Visual Literacy: A Conceptual Approach to Graphic Problem Solving. Watson-Guptill, New York, NY (2000)

We distinguish the dynamic aspect of the interaction gestalt as the nature of interaction from other types of gestalts. In Löwgren and Stolterman [A], a gestalt for interactive artifacts is defined as “dynamic gestalt” which “emerges in the interaction with the user over time (p.137).” This dynamic aspect of interaction has been emphasized by many researchers in different ways such as kinetic aesthetics focusing on bodily movement [B,C] and speed of artifact behaviors with a notion of “slow technology” introduced by [D]—i.e. creating a slow movement and transition of technologies which give time for reflections in order to create more aesthetic experience. All of them have emphasized the notion of time when describing the nature of interaction. The importance of the time aspect of interaction is also emphasized by [E,F,33]. The concept of the “dynamic gestalt” with the emphasis on time has influ-enced our definition of the interaction gestalt.” (p247)
(A) Löwgren, J. and Stolterman, E.: Thoughtful Interaction Design: A Design Perspective on Information Technology. MIT Press, Cambridge, MA (2004)
(B) Moen, J.: Towards people based movement interaction and kinaesthetic interaction experiences. In Proc. of the 2005 ACM Decennial Conference on Critical Computing: between Sense and Sensibility. ACM, New York. (2005) 121-124
(C) Svanæs, D.: Kinaesthetic Thinking: The Tacit Dimension of Interaction Design. Computers in Human Behavior. 13, 4 (1997) 443 – 463
(D) Hallnäs, L. and Redström, J.: Slow Technology – Designing for Reflection. Personal and Ubiquitous Comp. 5, 3 (2001) 201-212
(E) Davis, M.: Theoretical foundations for experiential systems design. In Proc. of SIGMM03. ACM, New York. (2003) 45-52
(F) Hallnas, L. and Redstrom, J.: From use to presence: on the expressions and aesthetics of everyday computational things. ACM TOCHI. 9, 2 (2002) 106-124
(G) Petersen, M. G., Iversen, O. S., Krogh, P. G., and Ludvigsen, M.: Aesthetic interaction: a pragmatist’s aesthetics of interactive systems. In Proc. of DIS 04. ACM, New York. (2004) 269-276

three key factors of interaction emerged as fundamental, namely time, space, and information. Time is already proved to be an important factor of interaction as we discussed above. Space is another key factor of interaction which we believe is critical. Although space is an important factor in any types of artifacts we design, we think that the way of conceptualizing space for interactive artifacts is quite unique comparing to other types of artifacts. It often connects physical and virtual spaces at the same time, and, even within a virtual space, the way of creating and feeling space is very different from what we do with physical artifacts. When virtual elements in an interactive artifact are combined with the concept of time—for example, movement, it also creates a kind of spatial perception even though it is only an illusion.” (p247)

An important character of these attributes is that they are not experience qualities. They are simply descriptions of the shape of the interaction, and not emerging experience qualities. Experience qualities are, as we discussed earlier, connected to personal judgment such as fun, engaging, comfortable, pleasant, excited, and etc., which do not describe the interaction shapes, but describe overall qualities of user experience.” (p249)

Our approach leads designers to think about interactions themselves without even thinking about the artifact properties. For example, if a designer considers movement as one attribute of an interaction gestalt he or she tries to design, he or she will explore a design space for the interaction gestalt from static to dynamic movements that can be shaped as an interaction both by a user’s input behaviors toward the artifact as well as by the artifact’s output behaviors shown to the user. Designers basically apply the attributes to manipulate both ‘input’ behaviors from users and ‘output’ behaviors through interactive artifacts. This process corresponds to visual designers’ manipulating the margin value from small to large, which is one attribute they deal with in the design of a 2-dimensional visual artifact.” (p250)

The most significant benefit of introducing this concept for aesthetics of interaction is that it enables designers to understand the effects of interactions themselves as their design target when exploring a design space. We believe that it will open up designers to think more clearly about the dynamic nature of interactions, and to explore various different forms of emerging behaviors over time through interactions.” (p250)

What designers explore with the idea of interaction gestalt is the space of emerging shapes of interactions; it is not about how interfaces look like or what features need to be implemented.” (p250)

fine the ideation of interaction gestalts. But attributes guide the conceptual directions of the intangible phenomena of interaction gestalts the designer wants to realize by manifesting them through the actual interactive artifacts. It is important to understand that the attributes are not supposed to be used individually. As the original meaning of gestalt tells us, the sum is different from the whole. The ways of combining attributes should be constantly explored and examined to establish useful interaction design principles, as we see from traditional design examples—e.g. various visual design principles such as juxtaposition, symmetry, contrast, and harmony.” (p251)

• Implementing design cases that apply these attributes for actual interaction design projects,
• Establishing a deeper understanding of the meaning of interaction itself from the design perspective.
” (p251)


**A Pragmatist Aesthetics Approach To The Design Of A Technological Artefact [Fiore, S.]

February 9, 2010

Taken from: A Pragmatist Aesthetics Approach To The Design Of A Technological Artefact
FIORE, S., WRIGHT, P. AND EDWARDS, A. (2005) A Pragmatist Aesthetics Approach To The Design Of A Technological Artefact, In Proceedings of the 4th Decennial Conference on Critical Computing: Between Sense And Sensibility, August 20-24, 2005, Aarhus, Denmark

Bibliographical Annotation

Fiore et al builds upon Petersen et al’s (2004) paper developing the dialogical experiential, embodied aspect of pragmatist aesthetics, and introducing ‘appropriation of objects as the aesthetic products of experience’ (pp129-130). Fiore et al see that with HCI adopting pragmatist aesthetics the discipline are opening up the scope of what behaviour and usability can be beyond what Don Norman defines as the first of three levels of beauty – the ‘visceral’ surface level (Emotional Design, 2005). Fiore positions Aesthetics as situational (a SITUATED ACTION?), an intrinsic resulting component of experience just as much as consummation and fulfilment. It is a reflective quality that emerges out of the experience, and only identifiable once the experience is exited. Therefore, they argue an object IS an event (p130). The meaning communicated by an object emerges out of how we use it, its social-cultural affordances and semiotic signifiers embodied in its form. Its significance and meaning is an appropriation through its creation, active use, and critique (p130). This aesthetical appropriation of an object therefore blurs the line between the user and designer, but connects the two in a creative act.

Useful Quotes

Aesthetics is a domain with an extensive genealogy which is reflected throughout the historical developments in design practice. In the context of this research, we are specifically concerned with the aesthetics of interaction, in which we see a broad distinction between the analytic, emphasising a view of humans as disembodied processors able to construct independent realities in the mind, and the pragmatic, which instead emphasises how people experience the world dialogically as embodied subjects. The critical difference between the two perspectives is in their legacy on our understanding of design: The former lends support to cognition as the foundation of interaction and a view of the designer as analyst aiming to meet objectively identifiable requirements in design, whereas the latter supports a more artistically-oriented idea about design, more able to account for the roles of emotion, engagement, the separation between objects-subjects and events unfolding unplanned as a normal feature of the instability of existence. Our preference towards pragmatist aesthetics emerges out of discontent with how approaches to design underestimate the relevance of fundamental characteristics of experience and understanding.” (p129)

A pragmatic aesthetics of interaction provides the basis for exploring an alternative conception of design based on understanding others (rather than interpreting observed behaviours or accepting propositional knowledge as certainty) and appropriation of objects as the aesthetic products of experience. Our work here involves theoretical and practical examination of how such an aesthetics can allow us to think more clearly about empathy in design. We explore a means for sighted designers to express their understanding of blind experience through the construction of artefacts and draw on appropriation, a fundamental process in aesthetic experience, as the basis for design empathy. Taking Dewey’s pragmatist aesthetics [A][B] and the related works of Shusterman [C][D] and Jackson [E] as a basis, we build on contributions seeking more holistic approaches to understanding and supporting experience in design [F][G][H] as well as others who reflect in design the details and subtleties of everyday life and blur the boundaries between
the ‘scientific’ and the ‘artistic’ [I][J].
” (pp129-130)
(A) Dewey, J. (1958). Experience and Nature. Dover
(B) Dewey, J. (1934). Art as experience. Capricorn
(C) Shusterman, R. (2002) Surface and Depth. Cornell
(D) Shusterman, R. (1992). Pragmatist aesthetics: Living beauty, rethinking art. Blackwell
(E) Jackson, P. W. (1998) John Dewey and the Lessons of Art. Yale Univ. Press
(F) Fiore, S. G. (2004). From designing for function to designing for meaning. In Proc. ECCE-12
(G) McCarthy, J and Wright, P. (2004b). Putting ‘felt-life’ at the centre of HCI. In Proc. ECCE-12.
(H) Petersen, M. G., Iversen, O. S., Krogh, P. G. & Ludvigsen, M. (2004) Aesthetic Interaction – A Pragmatist’s Aesthetics of Interactive Systems. In Proc. DIS2004.
(I) Dunne, A. & Raby, F. (2001). Design noir: The secret life of electronic objects. Birkhäuser.
(J) Gaver et al (2004) Cultural Probes and the value of uncertainty. In Interactions. September/October. 53-56.

Adopting a Pragmatist Aesthetics of Human-Computer Interaction prevents denigration of the senses to visceral satisfaction and is inconsistent with conceptions of interaction as the purely rational execution of routines. We are drawn towards an understanding of experience that holistically incorporates thinking, feeling, doing and effecting change within an intersubjectively constructed world: Experience has a consummation and fulfilment and an aesthetic quality which makes it intrinsically worthwhile. This quality is the indescribable sense we have of a situation: When we are in experience, we cannot describe it without first exiting it and transforming it into an object of reflection [A], but we may have a sense of the situation, a sort of immediate and pre-linguistic meaning or understanding that we feel [B]. However, when we encounter a situation that we cannot understand, we have to reflect on our experience and try to make sense of it. This may be reflection on second-hand propositional knowledge told by others or on something experienced directly for the first time. In this light, while understanding of an event when we are experiencing it may happen without conscious reflection on our understanding, making sense of the experience requires conscious reflection on the event. And this is central to our objectives here: An object is itself a form of event, with a unique past and future, whereby its meaning has been transformed through inquiry, enabling it to play a role in the conscious shaping of future experience [A]. As such, while we give meaning to events when we abstract and objectify them, that meaning may always change so that “an object is always an abstraction. It is like a sketch of the thing itself, a sketch in which certain features are highlighted and others overlooked.” [A, p25]. In the context of designing technological objects then, it is necessary to recognise the way in which the meaning we make of an object emerges both out of what we do [C] and the qualities an object embodies suggesting the potential for action and construction of meaning. Our interaction in the world thus enables us to construct meaningful experiences around objects [D] [E]. More importantly, as we select objects from our environment and give them meaning for the purposes of both utility and enjoyment [A], we appropriate them with respect for the historical significance they carry: We give things a meaning that no other person can and which we would not imagine for any other object in any other situation. The aesthetic of a technological object is then “a result of the human appropriation of the artifact… released in dialogue as we experience the world.” [F, p.271] so that an object achieves significance and meaning only when appropriated through active and critical reception and appreciated in its creation and use. The meaning of an object thus changes with respect to its history and significance. It is through an act of appropriation of the history of a thing that the perceiver is able to construct the meaning of an artefact as more than a functional object.” (p130)
(A) Jackson, P. W. (1998) John Dewey and the Lessons of Art. Yale Univ. Press.
(B) Dewey, J. (1958). Experience and Nature. Dover
(C) Dourish, P. (2001). Where the action is: The foundations of embodied Interaction. MIT Press.
(D) Petersen, M. G., Iversen, O. S., Krogh, P. G. & Ludvigsen, M. (2004) Aesthetic Interaction – A Pragmatist’s Aesthetics of Interactive Systems. In Proc. DIS2004.
(E) McCarthy, J. and Wright, P. C. (2004a). Technology as experience, MIT Press.
(F) Dreyfus, H. L. (2001). On the Internet. Routledge.

This appropriation requires a sensibility towards
the thing and its various levels of meanings. The aesthetic in the experience is thus rooted in the way in which this object is meaningful and transforms the perceiver’s understanding, making enjoyment so much deeper. Importantly, such notions of appropriation and construction of meaning do not imply merely interpreting an object to mean what we want, as this would deny all the enrichment and pleasure achieved from submitting ourselves to its alterity and seductive power [A]. The creative process is characterised neither by passive and irrational inspiration nor by a designer in full control of the productive process: these are instead both necessary and complementary moments of the experience of designing [A]. We at the same time bridge the gap between creating and perceiving by “reconceiving appreciation as creative production where the [perceiver] actively reconstructs the aesthetic object.” [A, p. 54]. The creative act is an experience which connects designer and audience. This has profound implications for our understanding of design. If appropriation of a thing involves actively constructing the aesthetic object and the experience, then the line begins to blur between the dualistic roles of designer and user. We can see a much closer and interdependent relationship between the experiences of creating and appreciating an object, because the very act of appreciating is itself constructive. Many questions consequently emerge regarding how we can realise the pragmatist ideals of design as a process of self-development, change, discovery and of reflection that is felt and sensed as well as intellectual.
” (p130)
(A) Shusterman, R. (1992). Pragmatist aesthetics: Living beauty, rethinking art. Blackwell

Each appropriation is unique and an opportunity for reflection by both the designer and others involved in the process, to find new meanings and possibilities within the emerging objects. Because of this, we see our current research as exemplary of one way in which appropriation may form the basis for an approach to design that is essentially exploratory and empathic. (…) In our process, appropriation is central to a series of creative phases involving designers, artists and engineers. (…) We can never know if our understanding corresponds with others’, but by adopting appropriation as our foundation in design, we respect and connect with the things others have already constructed out of their experience.” (p132)

***Aesthetic Interaction – A Pragmatist’s Aesthetics Of Interactive Systems [Petersen et al.]

February 9, 2010

Taken from: Aesthetic Interaction – A Pragmatist’s Aesthetics Of Interactive Systems
PETERSEN, M.G., IVERSEN, O. S., Krogh, P.G. and LUDVIGSEN, M (2004) Aesthetic Interaction – A Pragmatist’s Aesthetics Of Interactive Systems. In Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS). Cambridge, MA.

Bibliographical Annotation

Petersen, et al’s paper is often cited within HCI literature as a seminal paper that bridges the HCI research into aesthetics of interaction with the theoretical groundwork by Dewey and Shusterman on pragmatist aesthetics. What I began to find interesting is they see that using Dewey and Shusterman’s work on pragmatist aesthetics promotes the aesthetics of USE over APPEARANCE. This immediately resonated with me as my own thoughts on the repositioning of the influence of VisCom in IxD were leading me beyond the appearance into its heart. They state that in their paper they provide a framework perspective (p269) that is an alternative to other established perspectives.

Through a literature review they not only challenge the assumption of aesthetics (p270) being only about the visual impression, but also they challenge the HCI aversion to non-quantifiable factors that affect usability. This last challenge is something that both HCI has denigrated, and the VisCom disciplines have been too delinquent to address rigorously enough.

To set their framework they identify three central aspects of aesthetics: social-cultural, mind and body, instrumentality. (p270) In regard to their choice of pragmatist aesthetics they explain and justify their choice, seeing aesthetics as not a priori but as potential released through a experiential dialogue (p271). This aesthetic experience is experienced both cognitively and in an embodiment. This is an ambiguous experience (Cartesian Dualism? Spinoza? Epiphenominalism?). Aesthetic meaningfulness emerges from the experience of use rather than being predefined. It has an adhesive quality merging attractiveness and purposefulness in an interactive system (p271). It is not “added value” and more than surface “beauty”. It is dependant upon context and use, and quality and value comes from the experiential dialogue created through context and use. They draw upon embodiment in parallel with symbolic representations (semiotics of the interface). This embodiment, although they do not actually use the term, draws a connection to both Dourish and Suchman’s work (p274). [This is a connection to follow.] They position aesthetics as a fifth element of five interaction styles in Interaction Design. Important but aesthetics must address both human cognitive and embodied experiences, SITUATED within everyday life, and instrumental in SITUATED use (p275). The first four elements of interaction styles position the user as part of the SYSTEM (1), users being in control of the system (TOOL (2)), user and machine as equal partners in communication (DIALOGUE (3)), and IT as a mediator between human-human communication (MEDIA (4)) (p274).

Useful Quotes

A set of approaches are emerging each representing different applications of the terminology as well as different inherent assumptions on the role of the user, designer and interaction ideals. In this paper, we use the concept of Pragmatist Aesthetics to provide a framework for distinguishing between different approaches to aesthetics. Moreover, we use our own design cases to illustrate how pragmatist aesthetics is a promising path to follow in the context of designing interactive systems, as it promotes aesthetics of use, rather than aesthetics of appearance. We coin this approach in the perspective of aesthetic interaction. Finally we make the point that aesthetics is not re-defining everything known about interactive systems. We provide a framework placing this perspective among other perspectives on interaction.” (p269)

We seek to frame an extended expressiveness towards interactive systems through the concept of Aesthetic Interaction that can be obtained when the human body, intellect and all the senses are used in relation to interactive systems. However, when looking into the work that takes an aesthetic perspective on the design of interactive systems it becomes clear, that not all perceptions of aesthetics are equally fruitful as we see a danger in adopting superficial understandings of the aesthetics of interactive systems. We wish to challenge the assumption that aesthetics are mainly concerned with the immediate visual impression of products as we see it in e.g., [A], [B], [C].” (p270)
(A) Desmet, P. and Dijkhuis, E. (2003) A Wheelchair can be Fun: A Case of Emotion-drived Design. In Proceedings of DPPI’03. ACM Press, pp. 22-27.
(B) Fogarty, J, Forlizzi, J., and Hudson, S. E. (2001) Aesthetic Information Collages: Generating Decorative Displays that Contain Information. In Proceedings of UIST’01. ACM Press, pp. 141-150.
(C) Overbeeke, C.J., Djajadiningrat, J.P., Hummels, C.C.M. and Wensveen, S.A.G. (2002). Beauty in Usability: Forget about Ease of Use!. In Green, W.S and Jordan, P.W. (Ed.), Pleasure with products: Beyond usability, pp. 9-18, London: Taylor & Francis

as put by Overbeeke et al [A]. “Interfaces should be smart, seductive, rewarding, tempting, even moody, and thereby exhilarating to use” (A, p.10). We see two problems inherent in some of this work. First the assumption that users always want to have fun and be pleased represents a simplistic view on human nature.” (p270)
(A) Overbeeke, C.J., Djajadiningrat, J.P., Hummels, C.C.M. and Wensveen, S.A.G. (2002). Beauty in Usability: Forget about Ease of Use!. In Green, W.S and Jordan, P.W. (Ed.), Pleasure with products: Beyond usability, pp. 9-18, London: Taylor & Francis

We see a range of different applications of the same terms and more importantly these different applications represent different inherent assumptions about the role of users and designers (or artists) and interaction ideals. These inherent assumptions are well worth investigating when developing an aesthetic perspective on interactive systems design. For instance, we find that those who view the potential of aesthetics as the possibility to provide users with a pleasing visual appearance of products are leaving out much of the potential of aesthetics. To qualify the discussion on these matters, we draw upon the distinction made by Shusterman [A] between Analytic Aesthetics and Pragmatist Aesthetics. We argue in the following that Pragmatist Aesthetics is a strong theoretical basis to take on with respect to designing interactive systems and we provide examples of how we work to implement systems adopting pragmatist aesthetics.” (p270)
(A) Shusterman, R. (1992) Pragmatist Aesthetics. Living Beauty, Rethinking Art. Blackwell.

the very notion of aesthetic is used in ambiguous ways when it comes to answering the important question, what is the aesthetics of interactive systems. To answer the question we turn to pragmatic aesthetic as a theoretical foundation for staging a concept of aesthetic interaction. Shusterman [A] propagate pragmatic aesthetics as opposed to analytical aesthetics. We will use this distinction to qualify our discussion. Three central aspects of aesthetics will be discussed to establish a foundation for an aesthetic approach to interactive system design. These are the socio-cultural approach to aesthetics, designing for mind and body and the instrumentality of aesthetics.” (p270)
(A) Shusterman, R. (1992) Pragmatist Aesthetics. Living Beauty, Rethinking Art. Blackwell.

The analytic aesthetics in the words of Moore (1952) rely on the intuitive assessment of aesthetics of objects, as if the objects existed by themselves in isolation. In this analytic perspective, as the artist or designer shapes e.g. the chair of exquisite material, and aesthetics arise as a product property. Shusterman argues that until recently most analytic aesthetics simply ignored the socio cultural background as irrelevant, “probably because aesthetic experience was traditionally conceived as pertaining to immediacy, not only because of its immediate satisfactions but because of its assimilation to direct perception rather than inferential thinking.” ([A]pp.21). We see this perspective represented e.g. in works that assume that aesthetics of interactive systems can be evaluated based on visual perception of pictures [8]” (p270)
(A) Shusterman, R. (1992) Pragmatist Aesthetics. Living Beauty, Rethinking Art. Blackwell.
(B) Desmet, P. and Dijkhuis, E. (2003) A Wheelchair can be Fun: A Case of Emotion-drived Design. In Proceedings of DPPI’03. ACM Press, pp. 22-27.

In contrast, a pragmatic approach to aesthetics is represented by Dewey [A]. Dewey insists that art and the aesthetic cannot be understood without full appreciation of their socio historical dimensions. He stresses that art is not an abstract, autonomously aesthetic notion, but something materially rooted in the real world and significantly structured by its socio economic and political factors (A, pp.22). Accordingly, aesthetic is not inherent in the artefact itself but rather a result of the human appropriation of the artefact. Consequently, the chair is not aesthetic in itself but rather the aesthetic chair is a result of the socio-historical appreciation of the material, and the shapes. Accordingly our ability to engage in an aesthetic experience is based on our social context, manifested in a personal bodily and intellectual experience prolonged beyond the immediate experience. According to the thinking in pragmatist aesthetics, aesthetic is not something a priori in the world, but a potential that is released in dialogue as we experience the world; it is based on valuable use relations influencing the construction of our everyday life.” (p271)
(A) Dewey, J. (1987) Art as Experience. Carbondale: Southern Illinois University press.

Where as analytical aesthetics is preoccupied with separating humans into mind and body, a part for thinking and a part for sensing, pragmatist aesthetics insists on their inter-dependencies in the aesthetic experience. In a pragmatist perspective, aesthetic experience is closely linked not only to the analytic mind nor solely to the bodily experience; aesthetic experience speaks to both.” (p271)

According to pragmatist thinking the aesthetic experience encompasses the immediate sensational auditory, visual and tactile qualities of artefacts and the intellectual process of appropriating the artefact, and moreover it points to the fact that past experiences fashion those of the future. In a pragmatist perspective we have to move beyond ideals of meeting human sensor motor skills and somatic sensing, to include among others the human intellectual capacity to grasp and make sense of complex, contradictory and even ambiguous systems and situations [A].” (p271)
(A) Gaver, B., Beaver, J., Benford, S., Ambiguity as a resource for design. In Proceedings of CHI2003, ACM Press, pp. 233 – 240.

In a pragmatist perspective, for anything to have value it must relate to human needs, desires, fears and hopes. (…) They are appropriated in use. Meaningfulness and aesthetic experiences emerge in use, they are not predefined.” (p271)

What we stress here is that aesthetics has a purposeful role in the use of interactive systems, aesthetics is not only an adhesive making things attractive, and it is part of the foundation for a purposeful system. Aesthetics cannot be sat aside as an “added value”. Emerging in use; it is an integral part of the understanding of an interactive system, and its potential use.” (p271)

To summarize, a pragmatist approach to the aesthetics of interactive systems implies that aesthetics is tightly connected to context, use and instrumentality; circumscribing our perspective on Aesthetic Interaction. Thus it becomes meaningless to think of aesthetics of artifacts in themselves. They might contain an aesthetic potential, but its release is dependent on context and use. In Pragmatist Philosophy aesthetics is also released from its tight connection to art and its many definitions, instead it is connected to experiential quality and value. This provides the basis for focusing on the aesthetics of interaction related to our everyday experiential qualities when engaging in and designing interactive systems.” (p271)

Designing for aesthetic experiences invites people to actively participate in creating sense and meaning.” (p271)

In 1984, Bødker & Kammersgaard [A] reviewed different perspectives on human-computer interaction and coined four different but co-existing perspectives on interaction styles. Subsequently, these perspectives have been applied to provoke new design ideas through taking the different perspectives to the extreme in design brainstorms [25]. The four perspectives system, tool, dialogue partner and media (…) We do not wish to claim that these four perspectives on
design of interactive systems are no longer valuable, but we argue that these views lack the potential of addressing the experiential sides of everyday life. There are two main points to distinguish our fifth perspective from the four previous:
First, aesthetic interaction aims for creating involvement, experience, surprise and serendipity in interaction when using interactive systems (for further discussion see, Iversen, et. al. [C]). Whereas the dialogue partner perspective treats man and machine as equal dialogue partners, the aesthetic interaction perspective acknowledges man’s ability to interpret and appropriate technology. The ideal appropriation of technology is not the shortest way to mastery (as proposed by the tool perspective) but rather the process of appropriation itself becomes essential. Second, Aesthetic Interaction promotes bodily experiences as well as complex symbolic representations when interacting with systems. It puts an emphasis on an actively engaged user with both cognitive skills, emotional values and bodily capabilities.
” (p274)
(A) Bødker, S. & Kammersgaard, J. (1984): Interaktionsbegreber, internt arbejdsnotat, version 2.
(B) Monk, A. (2000) User-Centred Design. The Home use challenge. In Sloane, A. & van Rijn, F. Home Informatics and Telematics. Information, Technology and Society. Kluwer Academic Publishers, pp. 181-190.
(C) Iversen, O, Krogh, P & Petersen, Marianne G.(2003): The Fifth Element – Promoting the Perspective of Aesthetic Interaction, in proceedings of The Third Danish HCI Research symposium, Roskilde, nov 2003

We reasoned how the aesthetic experience through interaction relies on addressing both the mind and body, as well as it is rooted in the socio-cultural context of people’s everyday life. Moreover aesthetics in this perspective becomes instrumental to the use situation, going beyond ideas of “added value” and the immediate attractiveness of systems, placing aesthetics as an integral element of the artefact and a continuously encouraging element in the future use of a system. In order to exploit the full potential of aesthetics in interactive systems all three aspects has to be addressed. Working with this perspective of Aesthetic Interaction incorporates and highlights the experiential aspect of designing interaction. Although the aesthetic interaction perspective is important when designing interactive systems we position the aesthetic perspective as the fifth element of interaction design. Designing interactive system requires multiple perspectives.” (p275)

The concept of Aesthetic Interaction currently presents theoretical considerations and will need further empirical experiments in order to provide more concrete guidelines for working with aesthetic interaction generally. However we see Aesthetic Interaction as a beneficial perspective when designing interactive systems.” (p275)

Thoughts on Interaction Design [Kolko, J.]

February 8, 2010

Taken from: Thoughts on Interaction Design
KOLKO, J. (2010) Thoughts on Interaction Design. Burlington: Morgan Kaufmann

Bibliographical Annotation

This book, just published in 2010, is very informative in many ways regarding the development and position of IxD as a discipline. This is a very good, reflective resource but currently I have focused upon the text for specific information and frameworks to explore the aesthetics of interaction. Upon that subject, the text does naturally connect to the HCI community’s own research. This book is firmly in the Interaction Design literature (pp5,7,11), but naturally overlaps both HCI and Visual Communication. IxD is about behaviour, designing it and for it. It is a dialogue that is both emotional and physical, and is ‘manifested in form, function and technology’ (pp12-13). Within the text Kolko draws upon essays written by collaborators. Some of these essays are useful and some are quite weak. But this isn’t a book review.

One essayist, Chris Connors from Apple, discusses the distinction and usefulness of low and high fidelity prototyping (p36). In this section of his essay he discusses areas that Visual Communication plays an early part within the development process. Certainly Vis Com cannot own visual prototyping but certainly non-Vis Com team members use low and high fidelity techniques. These techniques for visualizing data draw upon techniques familiar to graphic designers. Although a tenuous link on the surface, this fact will later help my positioning in my research as a point of contact with the HCI research.

Kolko continues from Connors essay. Not mentioning by name the research area of ‘Aesthetics of Interaction’, he sees an IxDesigner connecting people through technology to the sensory, emotional qualities of data. This he sees these emotional qualities as generally pertaining to the aesthetics (p41) of ‘sensory data’. This phrase is interesting and I may return to explore this at a later stage. Through a discussion on the use of Personas, he equates scenario-based development as sketching of time, akin to visual sketching, as a problem-solving tool. This goes beyond mere visualizing, and into an activity to generate and discover solutions (p47). Abstract and semantic connections can be made through a plethora of visualizing techniques, revealing hidden relationships that need unpacking and understanding through reframing. Visualizing, or specifically diagramming using concept maps or process flow diagrams, synthesizes this information into further generative sketching/model making/prototyping (p64).

Visual form, regardless of the medium for the interaction involved, in Kolko’s view of IxD, is a basic method for communication (p78). This connects IxD to its sister design disciplines. The nature of an interactive artefact, both digital and tangible, means that part of its appreciation and understanding by a user results from affordances and semiotics. These factors are measured within a qualitative design framework. The quantitative disciplines, such as HCI, are moving toward trying to define a framework that would attempt to bridge the disciplines. This is the connection to the research area of ‘Aesthetics of Interaction”. The structure of an experience within an interaction needs understanding, reflection, and evaluation. Experience, and aesthetics resonate emotionally and experientially (p83), but it is easier for designers and audiences to critique and evaluate visual deliverables. The IxD discipline, like other design disciplines, would benefit from looking to its historical precedents, and grow from a gestalt rather than being reductive and assuming purity of disciplinary approach (pp88-89). The resonated with me as in my first paper I unpacked the role of the graphic designer within GUI design, and uncovered the precedent set by Modernists as being fluid to move across disciplines. This is a commonality within mainly qualitative design disciplines too, including IxD. Kolko states that an IxDesigner creates a ‘new visual form language’ where the Modernist design axiom ‘Form Follows Function’ is rejected. This ‘new visual form language’ Kolko raises takes into account that objects have more than the significance of functionality and the signifiers of form, but also an embedded semantic level, that is experiential and emotional (p102). Within this new design space between form, function and experience an interaction takes place.

This conceptual space is physical, cognitive and social (p104), and I would look to explore this further as I believe the HCI research into Pragmatist Aesthetics could also be located in this ‘conceptual space’. The danger is assuming too early that both map onto each other. As semiotics can be divided into the branches of both semantics and pragmatics, it is within this framework that aesthetics can also be discussed. Kolko also draws upon Dewey (p108), who in turn influenced Shusterman’s Pragmatist Aesthetics. As with HCI researchers, IxD designers equally can find Dewey’s Pragmatist philosophy of conceptual use. As a pragmatist view also resonates with Dourish’s theory embodied interaction, a potential conceptual conduit to connect it all together may be Csikszentmihalyi’s ‘Flow’ (p109).

Uday Gajendar, another essayist within the book (who wrote the paper ‘Experiential Aesthetics: A Framework for Beautiful Experience’ I have found useful), adds to the ‘conceptual space’ experiential discourse. He sees the design of/for interaction as a situated activity (pp114, 120), which makes connections to Suchman’s theory, and of embodiment (Dourish) (pp116, 120). Again another designer making the connections between the (until now separated) areas I had been exploring when I began looking into a Designer’s Model. I appeared to have had the sources but not, until now begun to see the connections.

Useful Quotes

There are, however, few texts that explore the semantic connections that live between technology and form which are brought to life when someone uses a product. These connections may be thought of as “interactions” or “experiences”, and are becoming to hint that a field known as Design (with a capitol “D”) is a legitimately separate area of study alongside Science or Art. This text” (p5)

Interaction Designers are trained to observe humanity and to balance complicated ideas, and are used to thinking in opposites: large and small, conceptual and pragmatic, human and technical. They are shapers of behavior. (…) the value of Interaction Design (is) in the development of human-centered designs and in the creation of a framework in which to experience these designs.” (p7)

Interaction Design is the creation of a dialogue between a person and a product, service, or system. This dialogue is usually found in the world of behavior (…) Structuring dialogue is difficult, as it occurs in a fourth dimension – over time. To design behavior requires an understanding of the fluidity of natural dialogue, which is both reactionary and anticipatory at the same time.” (p11)

Design work is of function, and language, and meaning. Through visual and semantic language, a designer must create a design that assists the viewer not only in experiencing a particular emotion but also in truly understanding the content. (…) Interaction Designers, however, speak both words and form at once. They structure a compelling argument and invite the audience to share in their work. The work evolves over time, and the work is completed by the presence and synthesis of the audience.” (p11)

Interaction Design is the creation of a dialogue between a person and a product, service, or system. This dialogue is both physical and emotional in nature, and is manifested in form, function, and technology. (…) This is the purpose of the profession: to change the way people behave (…) attempt to ease the suffering of their end user.” (pp12-13)

to reposition the the field of Interaction Design away from a solely technical field or an artistic endeavor, and instead towards a duality that emphasizes the human side of technology.” (p13)


Those engaged in HCI activities – Interaction Designers – exist to ask these difficult questions, and to create frameworks for compelling experiences rather than technical experiences. Interaction Design has outgrown its computing roots, and is now a field responsible for humanizing technology (…) to tame the complexity created by technological advancements.” (p#)

process drives repeatability – reducing the reliance on inspiration – and creates a framework in which creative professionals can execute. (…) Most designers are used to employing a design process, most likely the one employed by their source of design or studio training (…) to adopt and standardize a single process, define the range and types of deliverables for each process phase, and then, consistently apply it.
” (pp31-32)

It’s not uncommon to find a method or algorithm that initially seemed viable in practice might not scale adequately or offer the required performance, causing engineers to have to re-think their approach. [sic] Should designers find themselves in a state where they need to re-assess assumptions, use this common ground to build rapport, assess schedule impact, and move on.
” (p32)

What agile methods offer designers are an opportunity to design the product in a broad sense, and then the chance to execute designs in manageable sections over the development cycle. Designers may have to do some selling in order to convince developers to afford them some time up front to get ahead of the development cycle
” (p33)


Prototypes can be high or low fidelity visually – hand drawn vs pixel accurate renderings. The navigation can be high or low fidelity in terms of breadth or depth. They can also have high or low fidelity interactivity, and perhaps most importantly, high or low fidelity data, where high fidelity data might represent an actual data set and low fidelity data might be a few spoofed data elements – “loren ipsum” rather than actual text, for example. (…) A low visual fidelity prototype with high fidelity depth can help evaluators elicit user responses to an entire process through an artifact.
” (p36)


the field in and of itself has nothing inherently to do with computers. Instead, the field is best thought of in terms of a methodology, and the major contribution an Interaction Designer can provide in a business setting is a strong process that connects people, technology, and the emotional qualities of sensory data (generally pertaining to aesthetics).” (p41)

The process of human-centered design relies havily on modeling target users in an effort to create a prototypical audience for design. A model is a representation of a real thing, and a model of a user is a representation of a real person. A basic form of model that has been embraced by Interaction Designers and is created in the initial stages of a project is the Persona. (…) The Persona begins to become an active member of the design team.” (p45)

Traditional user-research can, and should, inform the creation of the Persona. (…) A good Persona is rich with detail and is thus predictable, in the same way that one can predict the actions of a friend or loved one. While these predictions may not be right all the time, it is possible to anticipate with some degree of accuracy what an individual will do in a given situation.” (p46)

The use of scenario-based product development has several core benefits. (…) As behavior exists in the fourth dimension, these scenarios become sketches of time. Industrial Designers and Graphic Designers can quickly explain the value of visual sketching in their design process: Sketching is a problem-solving tool, used not simply to visualize ideas but to actually discover and generate a large number of solutions to a problem.” (p47)

A design solution is judged based on the relevance to the individual who ultimately must use the creation.” (p48)

Participant observation is an important aspect of Interaction Design, as it formally acknowledges that a product does not exist in a rational and substantial way until it is considered by an audience. (…) The product needs to fit appropriately into the culture in which it is to be used and sold.” (p49)

People have a very hard time explaining why they do the things they do, and human behavior often seems illogical when considered by an impartial observer. Therefore, interpretation – making meaning of gathered data – plays a critical role in translating research into valuable design criteria.” (p50)


Focus is the acknowledged pre-set view of what is going to be addressed through the ethnographic inquiry. It gives the designers a central topic to attend to and a statement to rally around. (…) A focus statement takes the conceptual approach of framing an enquiry.” (pp50-51)


To understand context, go to the place where work occurs: Go to the users, rather than bringing the users to you, and watch what they do as they conduct real work. So simple, yet so evasive.” (p51)

People do strange things – unexpected things – and being there to witness and record these minute and quick moments of humanity is simply invaluable to the product development process. These details trigger design insights, and the equally important rationale to back up the design decisions to other members of the design team.” (p51)

As the goal of a Contextual Inquiry is to gather as much rich data as possible, it is important to reject this logic and become an active participant in the inquiry. This participation takes the form of partnership, and is likened to that of a master and apprentice in the days of guilds. An apprentice did not sit quietly and observe. He became engaged, and tried things, and questioned things, and assisted in the process.” (p52)

Experience is a guide to better understand when to ask questions and when to remain quiet, but a master and apprentice relationship will allow an investigator to best understand the nuances of work and truly gain the confidence of the participant being observed. Interpretation, or the assignment of meaning to fact, is a subjective form of synthesis. It is also the most critical part of the Critical Inquiry process, and the portion of the process that is ignored with the most frequency. (…) Interpretation occurs in context, but the critical interpretation often occurs back in the “lab” – in the design studio, while the designer is sketching. [] Interpretation is qualitative, and can be wrong. This makes for a diificult combination when trying to justify design decisions.” (p52)

Frequently, interpretation occurs in the head of the designer. This “moment of epiphany” may be thought of in the shower or scrawled on the back of a napkin. The Interaction Designerunderstands the importances of structuring this interpretation into a repeatable and formal process, and a good Interaction Designer is able to communicate not only the pragmatic interpretation but also the necessity of interpretation.” (p52)


These phases, while highly intellectual, also require the “designer’s intuition” and frequently rely on rapid ideation sketching, additional narrative development, and mind mapping as a generative method of problem solving and concept development. (…) a designer works through both a convergent and a divergent thought process of ideation. Convergent thinking attempts to locate the best answer (…) Divergent thought implies a great deal of risk. One must shift perspectives away from the safety of familiarity in order to explore what “could be”.” (pp54-55)


Design is a creative field, and in order to successfully create, one must achieve a sense of Flow. Flow is, among other things, the absence of self-doubt and the nearly auto-telic and automatic creative process.” (p58)

At the core of an interaction is the dialogue between a product, system, or service – and a person. Design exists as a means to a greater end – enhancing the human experience, solving complicated problems and ultimately creating designs that resonate with their audience.” (p62)

During the process of design, the Interaction Designer attempts to construct meaningful visualizations between individual components in an effort to understand hidden relationships. The ultimate goal of the creation of these visualizations is to understand. By reframing ideas in new and interesting ways, the designer can gain a deeper understanding of the abstract and semantic connections between ideas. These visualizations can then be used to communicate to other members of a design team, or can be used as platforms for the creation of generative sketching or model making. Frequently, the act of diagramming is a form of synthesis, and is a way to actively gain knowledge.” (p64)


Design literature frequently mentions a four-step process taken as individuals gain comprehension. (…) This four-step process attempts to move from Data to Information, to Knowledge, and finally to wisdom (DIKW). (…) Interaction Designers can think of this DIKW path as a framework for progressive learning. (…) Making information out of data, a seemingly easy task, is quickly confounded when the designer attempts to integrate elements of aesthetics or emotion. (…) Information is the organisation of data in ways that illustrate meaning. This organization may, in fact, alter the meaning itself. This has an important implication, as the meaning of seemingly objective data is altered by the appearance and structure of that data.” (p65)

While information may be sensory, knowledge seems to be more complicated, and perhaps more experience-driven. Storytelling has a long history as a mechanism of knowledge transfer, and can be considered a rapid immersion in experience. (…) This idea of knowledge as extended dialogue is highly relevant when considered in the guise of experience and Interaction Design. The design of behavior may, in fact, be the design of action-based knowledge (telling a story through motion).” (pp65-66)

Wisdom, often thought of as enlightenment, can result from applying knowledge in a new and novel manner. (…) The acquisition of knowledge obviously occurs over time, and this is where the Interaction Designer excels. Behavior occurs in the fourth dimension, and Interaction Design techniques attempt to understand and, hopefully, shape the way people act over time.” (p66)


The Interaction Designer (attends) to the detail and pragmatic details of UI design only after modeling or understanding the more conceptual behavior – activities or goals – that may drive the usage of a product. Several mapping and diagramming techniques exist to assist Interaction Designers in tracking product use over time. While referenced by various names in various disciplines, they all attempt to create systematic organization amidst complexity.” (p67)

A concept map is a visualization of present understanding of a system. It is intended to represent the mental model of a concept – to allow members of a development team to see the “forest and the trees”. Generally, a concept map links nouns with verbs. It provides a visual way to understand relationships through literal connections as well as through proximity, size, shape, and scale. The tool is intended to illustrate relationships between entities. The act of creation is generative in the sense that the designer must make subjective value judgments on the strengths of relationships. The first step towards creating a concept map is the creation of a concept matrix. This matrix lists all elements relevant to a particular domain (nouns) and attempts to identify which items have a direct relationship. (…) By creating a matrix to illustrate the connections between these elements, the designer is forced to analyze the extent of the relationship. (…) By analyzing each and every term’s connections to one another, the designer is forced to “zoom in” on the details to such an extent that he gains an intimate understanding of a discipline. He can then begin to understand the (sometimes obvious) hierarchy that exists within a large quantity of data. The elements with more relationships become the main branches on the concept map. (…) Once the matrix is created and these core concepts are identified, completing the concept map becomes a rather simple activity of connecting nouns with verbs. (…) As these are added to the diagram, the designer – and eventually, the entire development team – can visually trace relationships between entities and understand how a potential change to one aspect of a system may ripple through the system as a whole.” (p68)

Process Flow Diagrams are another visual form of organizing data into comprehensible systems. Also known as Data Flow Diagrams or Decision Tree Diagrams, these diagrams have traditionally been used in the fields of electrical engineering and in computer science to illustrate the logical flow of data through a system. To create a Process Flow Diagram, an Interaction Designer first identifies, through various forms of ethnography, the operators in a system and their roles. These operators include many of the nouns as present in the Concept Map. Then, the “logic flow” is mapped out to connect the operators with actions. (…) By creating a Process Flow Diagram, the designer has formed an intimate understanding of the possible logical outcomes of use with a system. While the diagram itself can be useful throughout the project, the act of creating the diagram is of much more importance. Those involved in the production of such a diagram have created a strong mental representation of the boundaries of a complicated system.” (pp68-69)

Both of these aforementioned diagrams embrace the visual over the textual. While they certainly include written words, the visual arrangement of the content creates an arguably more accessible way of examining a system or artifact. The diagrams rely on the use of words as placeholders for ideas, forms or artifacts. language affects organization – and therefore, usability – on a very pragmatic and immediate level. Categorization implies the method that is used to group elements within a larger context. People rely on language in design to encourage simplicity, yet language is often ambiguous and many designers are not adequately trained in the nuances the English language presents. (…) Designers, then, must understand the trivialities associated with the words they select for everything from the labels on a website to the packaging an object comes in.” (pp69-70)

The Interaction designer attempts to construct meaningful visualizations between individual components in an effort to understand hidden relationships. The ultimate goal of the creation of these visualizations is to understand; by reframing ideas in new and interesting ways, the designer can gain a deeper understanding of the abstract and semantic connections between ideas. This understanding can then be applied to the development of a system, service, or artifact.” (p71)


Designers are in the unique position to improve all aspects of human life, including the visual, emotional, and experiential. Interaction Design should be desirable – beautiful, elegant, and appropriate – regardless of the medium chosen to visualize a solution. Visual form can be considered one of the most basic methods of communicating design solutions.” (p78)

Rhetorical issues of form development become increasingly important when considering solutions that embrace technology, as ambiguity of form may negatively impact understanding but may positively affect experience. Many Interaction Designers are deeply concerned with the nature of aesthetics, continually considering why objects look the way they do and analyzing the relationship between particular cultural movements brand identity “formulas”, and trends.” (p78)

Interaction Designers work with (or as) graphic or visual designers to establish consistent sizes, placements, shapes, colors, and styles in order to continually reinforce brand language.” (p81)

Interaction Designers do not consider a designed artifact as distinct from the experience in which it is found.” (p83)


As aesthetics and experience are so closely related, it is important to evaluate not only the emotional or experiential resonance in the creations, but also to understand or contemplate the structure of experiences with artifacts. (…) Experience itself occurs (probably continually) during moments of consciousness, as to experience the world or to consider what is occurring in the world at a given moment. (…) Experience as story is the vehicle used to transmit, condense, and reflect on an experience. (…) designers are more fruitful in focusing their efforts on the creation of the structure in which an experience takes place.” (p83)

Interaction Design should be desirable – beautiful, elegant, and appropriate – regardless of the medium chosen to visualize a solution. And while the aesthetic refinement is important to the success of a product, the ability for that product to resonate in an experiential manner will allow that product to remain embedded in and positively affecting society and culture.” (p84)


In industry, we put up walls between ourselves and between our clients. We like to classify each other and characterize design as “industrial” or “graphic”. But the discipline be damned: it is our ability to think creatively and broadly, not our physical output (be it words, renderings, or diagrams) that defines us professionally. The focus on a designer as stylist – on the visual aspect of design – is not surprising. The visual is our tangible deliverable, and appears to be our greatest (and only) contribution. It is far easier to “critique” and evaluate the physical characteristics of a product rather than debate the products experience or emotional benefits; we concentrate on the “prettiest” picture instead of the best solution. Designers are traditionally labeled as the “makers” of “pretty things”, and as such, we exist at the end of a long process – not where we belong – at the beginning. This placement forces us into a predestined flow dictated by the establishments of marketing, technology, and aesthetics. (…) Our industry and our educational system are both to blame. We both focus considerable time on creating the tangible instead of the intellectual. (…) Professionals spend the majority of their time competing on the level of “cool” instead on the level of “thought”. This battle to create the most “bling” is detrimental to designers, to design, and to our clients. Our inability to articulate the importance of process means our clients focus on “money shot” renderings while they overlook the basic testaments of user centric design; moreover, as project managers equate design to “pretty pictures”, they gloss over the true usefulness of the discipline: innovation and differentiation.” (pp85-86)

Design is typically described as a visual discipline. However, that is only partly true. Our discipline has historically welcomed disparate professions into our fold like computer scientists, researchers, cognitive psychologists, and business analysts. Globally, however, we tend to forget that this is a historical precedent, not a trend. As such, we should embrace their best practices and processes to achieve successful communication of our visions.” (pp88-89)

The Interaction Designer shapes culture directly through the creation of new visual form language. This semantic view of design – that objects are embedded with more than just functional significance – rejects the platitude of Form Follows Function and instead recognizes the need for emotional and social connections in the human-made world.” (p102)


While there certainly is a market for “cool things”, some designers find the emphasis on styling and visual aesthetics as superficial – a great deal of the design community feels that the designer provides a deeply intellectual contribution in the creation of the goods, and the sensory elements are only the most immediate “hook” for people to respond to a creation. In fact, there is much more substance to designed artifacts, and it is this substance that allows them to resonate in a meaningful fashion. This substance is what Saussure viewed as the linguistic sign, what Evenson and Rheinfrank viewed as a design language, and what Buchanan considered as the harmonious combination of rational, human, and stylistic. (…) An interaction occurs in the conceptual space between a person and an object. It is at once physical, cognitive, and social. A poetic interaction is one that resonates immediately but yet continues to inform later – it is one that causes reflection, and one that relies heavily on a state of emotional awareness. Additionally, a poetic interaction is one that is nearly always subtle, yet mindful.” (p104)


Author, psychologist, and philosopher John Dewey explains that “Experience does not go on simply inside a person. (…) Every genuine experience has an active side which changes in some degree the objective conditions under which changes in some degree the objective conditions under which experiences are had.” [A] This implies that, while an Interaction Designer may focus on the creation of an artifact or system, much of the “meat” of the experience of use is left up to the person using the artifact or system.” (p108)
(A) Dewey, J. Experience and Education, Free Press, Reprint Edition, 1997, p39


Mihaly Csikszentmihalyi has been analyzing the essence of creativity, and has identified the state of being known as “flow” to be one that encourages a vivid awareness of the moment but an almost lack of awareness of the surrounding environment and task. As Csikszentmihalyi describes, during flow, the sense of self and self-consciousness disappears. (…) Perhaps, then it is useful to attempt to recall not a particular interaction but the beauty of the associative scene.” (p109)

To resonate poetic, the interaction one has with a product should be engaging, appropriately complicated to the given task in order to encourage a mindful state, and highly sensory. But it is important to note that the moment need not be long.” (p110)


There is no question our landscape of human experience has become over-populated with varieties of artificial (increasingly digital) content, (…) all vying for for someone’s attention. (…) Yet each form is an invitation for a personal encounter to interact and thus play, share learn, or create, within a specific context – hence, the emergence of situated moments. Each moment involves multiple layers of sense-making and discovery, as the user perceives and interprets the form, functionality, and style – in other words, the design.
” (p113)

Interaction is a generative, constructive phenomenon among a live being, an artificial form, and a context, influencing one’s quality of experience, and facilitating the transference (or mutation) of meaning from the designer to the intended user, as mediated by the product’s qualities and features. Accordingly, a design is not merely stylish, attention-grabbing ephemera but a vital form of discourse augmenting (or detracting) the cultural (and experential) landscape in which we live and thrive.
” (pp113-114)

In its purest form interaction refers to a dynamic relationship between reciprocating entities at varying types and degrees of influence: people, environment, natural forces, and spiritual/cultural ideas.
” (p114)

Design, in this case, means the conception, planning, and making of “the artificial” (products, services, systems, environments) that serve individual and collective human goals. It is a situated activity, dependent upon the circumstances of use (as well as the conditions of product development). It is also a deeply human enterprise, contingent upon personal skills in imagination, empathy, synthetic thinking, and visual communication.
” (p114)


The human conceptual cognitive system is “fundamentally metaphorical in nature”. (…) So, metaphors are basically conceptual aids to understand abstract entities in terms of concrete objects, thus helping people make sense of the complex, dynamic surroundings. Much of this is predicated upon the “embodied mind” notion of human bodies (and almost symbiotically connected mind) having physical experiences in a spatial orientation, which affects the perception of reality accordingly from that viewpoint. (…) Thus, a metaphor operates through a mapping of conceptual domains, to facilitate the interpretation of the unfamiliar in terms of the familiar.
” (p116)


Paul Dourish has hypothesized a different take on Interaction Design, that he terms as “embodied interaction”, a new model of interpreting interaction that extends recent HCI research trends in “tangible” and “social” computing. Dourish’s argument is based upon the philosophical framework of phenomenology which is the study of experience and existence, that are intuitively felt and known by factual presence in the world. Dourish contends that embodiment is more than a physical property but is about social presence and participatory status in the world, having an (inter) active role in changing and becoming. Everyday engagement in daily activities and task completion is another core tenet; the setting of action defines the value and manner of the action. Thus meaning emerges from the participation of an individual agent with some object within a setting – a constant negotiation or conversation unfolding. It is formed continuously and interactively, in real-time action/location; meaning is not simply projected or found but instead created and shared through engagement with the artificial. This is a profound view of interaction that shifts the emphasis from the designer crafting the argument, or the interpretation of images, towards the place of action between the user and the object in question, given a situation and the particular lifestyle of the user. This view encourages the designer to regard design as a participatory activity, not simply dictating to the user, but allowing the user to evolve and shape the encounter so it is a co-operative opportunity.
” (p120)


In guiding the designer who seeks an effective communication-oriented solution, these views parcel out finer issues for debate and iteration. These are simply ways to perceive how meaning comes to be in interaction, when regarded as a communicative activity. In actual practice, however, an interactive encounter (and thus meaning itself) combines all three views into a dynamic, self-sufficient, whole user experience. (…) A coherent and consistent system of interactions within the framework of design suggests a language of relationship building between people (user + designer, user + other users) mediated by the designed artifice. Value and meaning are deliberated, interpreted, and created via the interactive encounter, at multiple levels: emotional, cognitive, physical, visual.
” (p121)


Interaction Designers is not about a transient aesthetic. (…) Interaction Designers are trained to observe humanity and to balance complicated ideas, and are used to thinking in opposites in opposites: large and small, conceptual and pragmatic, human and technical. This is not a jack of all trades. Instead, it is the shaper of behavior. Behavior is a large idea, and may at first, seem too large to warrant a single profession.” (p151)

Digital Ground [McCullough, M.]

February 6, 2010

Taken from: Digital Ground
McCULLOUGH, M. (2005) Digital Ground: Architecture, Pervasive Computing, and Environmental Knowing. Cambridge: MIT Press

Bibliographical Annotation

Amongst the main theme of McCullough’s thesis about situated, embodied interaction through objects rather than desktops, I find that there is an additional flow of ideas (that supports his thesis) of interest. In exploring the exploration of embodied interactions situated within context, space and place, McCullough connects several areas of thought that until reading the book I hadn’t fully connected.

To support his thesis his argument flows progressively through: spatial relationships (p33), mental models (p33), embodiment in context (p36, pp49-59), situated actions (p52), embodied cognition (p55), beyond the desktop (p69), representation of contexts (p97), beyond visual identity into experience (p157), cultural assimilation (p168), flow (p191), value (p194), towards aesthetics (p196). His thesis goes beyond these arguments, but for me they provide me with a framework in which to explore my own thesis.

Originally it was to explore a new interpretation of a cognitive “Designer’s Model” for Interaction Design. This led me to a dead-end within computer science. In researching this subject I independently began reading within specific areas of cognitive psychology around conceptual models and embodied cognition. This led to Suchman’s “Situated Actions”. Before this I had set a context to where the Visual Communication literature stood theoretically, practically and historically on interaction. My in-articulated inkling was that to regenerate Vis Com’s (especially Graphic Design) influence on Interaction Design would take me beyond the desktop into the contextual factors of experience.

This ‘context-centred design’ suggests potential relationships between Visual Communication and Interaction Design in closer fundamental ways. As some movement within HCI research exists towards quantifying aesthetics as an important component that aids usability, McCullough’s framework flow suggests a tendril of arguments that may span between the qualitative and qualitative disciplines, allowing me to re-ground Vis Com into a stronger theoretical and practical position within Interaction Design.

Useful Quotes

embedding information technology into the ambient social complexities of the physical world” (pX)

The saturation of the world with sensors and microchips should be a major story, and an active concern for designers.” (pXIII)

Digital systems that are carried, worn, and embedded into physical situations can fundamentally alter how people interact.” (pXIV)

information technology must be moved from the centre of our focal attention into the peripphery, and conversely, how certain contexts become responsive through the addition of technology” (pXIV)

Generally as information becomes more and more abundant, clear views through it become less and less possible.” (p15)

No longer just made of objects, computing now consists of situations.” (p21)

Embodiment is not just a state of being but an emergent quality of interactions.” (p27)

For interaction designers seeking to know more about context, space and place (…) the principles of embodied predispositions provide increasingly common ground.” (p27)

The exploration of embodied interactions reveals to us conditions otherwise taken for granted.” (p27)

Mental attributes and constructs are emergent (…) Thus the structure of embodiment, itself a product of adaptation to environment, may underlie emergent intent. (…) Bodies shape conceptual structure; environmental experience grounds metaphor; and a lot more thought is metaphorical than has been assumed previously.” (p32)

Apparently humans assimilate their surroundings by means of mentally constructed representations of spatial relationships. Formerly, researchers held that such environmental schemas are purely mental, but now there is greater recognition of direct engagement and peripheral awareness as compliments to deliberate mental models.” (p33)

Contexts do not induce actions so much as shape perceptual selectivity, provide background cues, and enable the application of tacit knowledge. Active embodiment cues what would otherwise be isolated sensory awareness. Intent in context causes cognition to be about something. ” (p34)

Atop a continually changing substrate of embodied perception, the abstract mental model arises only occassionally, and only when necessary.” (p34)

Contextual learning begins as embodiment, remains largely personal, and is life long.” (p35)

Because contexts are learned through actions and events, much of this understanding is based on memories of interactions: object permanence, landmarks, proportional configurations, spatial categories, procedural contexts, swapped frames of preference, geometric measures, building elements, generative typologies, systemic behaviours, formal elegance, regional characteristics, ecological sustainability.” (p36)

It is important to note that embodied learning occurs at several levels, ranging from preconscious engagement of affordances, to the personal construction of mental models, to the cultural mediation of spatial literacy.” (p36#)

Contexts are full of props and cues, which serve as learning resources and memory devices for evolving patterns of usage. Many such cues serve as constraints, context rules some things out so that others may receive closer attention. Those perceived resources are appropriated toward and active intent. This grasp is engaged but not necessarily reflective. It is as much a product of the abilities and intents of the subject as of the properties of the object. This is one reason why the use of tools transforms the perception of environment.” (p37)

Language itself plays an important role in spatial literacy. Language abounds with bodily metaphors that recall the experience of environment.” (p37)

Compared with some rather more difficult social conditions, attention to embodiment provides a fairly straightforward opportunity to develop the expression and valuation of properties that for too long have been dismissed as unmeasurable.” (p44)

The disciplines of architecture and interaction design both address how contexts shape actions. Architecture frames intentions. Interactivity, at its very roots, connects those mental states to available opportunities for participation. These processes are ambient. Their benefits are to be found in the quiet periphery, and not in the seductive objects of attention.” (p47)

most agendas of physical computing share a belief in “periphery.” As defined by John Seely-Brown, the former director of (…) Xerox PARC, “periphery is background that is outside focal attention but which can quickly be given that attention when necessary.” This is one way to deal with information overload. “Periphery is informing without overburdening.”” (p49)

Principles of periphery can help reduce contention on a screen, of course, but they also suggest a larger shift in our goals for natural interactions. This is mainly a matter of embodiment in context.” (p59)

As reflected by so much recent emphasis on embodiment, contextual factors matter more than early researchers in interactivity anticipated. If more recent study finds the phenomenology of engagement at the roots of interactivity, it is because these designers build technologies around everyday life. This shifts design values from objects to experiences, from performance to appropriateness, from procedure to situation, and from behaviour to intent.” (p50)

Cognitive science has emphasized mental representations at the expense of context. [Bonnie Nardi] “Thus we have produced reams of studies on mentalistic phenomena such as ‘plans’ and ‘mental models’ and ‘cognitive maps,’ with insufficient attention to the world of physical artefacts.” Designers more interested in rich description than in predictive models tend to welcome such emphasis on artefacts. As a way of describing the intrinsic unity of context, activity, and intentionality, “activity theory” has become a useful expression.” (pp50-51)

[Lucy Suchman] “The organization of the situated action is an emergent property of moment-by-moment interactions between actors, and between actors and the environments of their action.” Within the situated action model of work, actors operate within a stable institutional framework, or “arena,” to create personally ordered versions of the environment matched to their respective habits or goals. Habitual contexts support courses of action in which effectiveness has been internalized enough that it need not rise to the level of a conscious mental model. For example, a competent intern makes hospital rounds according to a well-established procedure, but an expert doctor makes his or her rounds according to more tacit and personalized criteria.” (p52)

Accumulated experience of intent and action allows more abstract mental models to develop.” (p53)

Through context, designed objects become expressions of identity, signs of differentiation, tokens of communication, and a natural support for relationships.” (p53)

A theory of place for interaction design incorporates embodied cognition into a workable design philosophy through types. In a single design notion, type unites periphery, passivity, phenomenology, adaptability, affordance, facility, appropriateness, and scale. (…) For present purposes, consider type not as a mere functional classification, but as a generative design abstraction. This is a central idea for context-based pervasive computing (…) A type may be as much about form as function.” (p55)

[Interaction Design] takes advantage of physical contexts as frames and cues for its social functions. It begins to reflect scale and type in its pursuit of site-specific technology, context-aware systems, and location-based services. it shifts focus from technological novelty to more enduring cultural frameworks.” (p63)

For any new approach to design to break out of this feature accumulation cycle, information technology must change fundamentally, that is, at a level much more basic than a better desktop interface.” (p69)

The next stage of evolution takes the load off a technology now two paradigms old. Thing centred computing is coming to be for the 2000s what network-centred computing was to the 1990s and personal computing was to the 1980s.” (p69)

Physical devices establish possibilities for interaction beyond the desktop. Local models are necessary abstractions for technology-extensible places. Social situations provide design precedents and problems from which to build types. All of this points toward new forms of context-centred design.” (p72)

Even at a purely technological level, location still matters. Location models prove essential for pervasive computing. (…) This is basically a question of representing action. (…) How much of that can be modeled, and how much must remain implicit?” (p97)

As long as the desktop remained the stage for information technology, location models seemed almost irrelevant. Indeed for a while many people seemed willing to take the (metaphorical) representation for the (virtual) reality. However, continued human expectations for embodiment and periphery have turned the tide. As we now take mobile devices out into the physical world, and increasingly bring them into contact with intelligent environments built from embedded systems, our digitally mediated actions truly must take place somewhere. The representation of contexts now becomes the essential challenge to designers of information technology.” (p97)

To an architect, a model chiefly represents form, but to other disciplines, a model may represent behaviour, information flows, or decision sequences.” (pp97-98)

Our very presence in one kind of space must serve as consent to take part in its technical environment, but in another space should indicate our desire for anonymity. (…) These are questions of how to present oneself. Traditionally such questions are answered by social customs. (…) Each of these factors point towards the need for spatial modeling of digital mediated action. In contrast to the usual assumptions about formless dematerialization, the rise of pervasive computing restores an emphasis on geometry.” (p101)

human interactions continue to exhibit categories, strata, and patterns.Such recurring configurations are natural; just about any species has them. Contexts remind people and other devices how to behave. That framing has often been done best and understood most easily as architecture. Something about the habitual nature of an environmental-usage gives it life. Like device protocols and personal conduct, architecture has been a form of etiquette, architecture exists not out of pompousness, but because it lets life proceed more easily. Situated computing extends this age-old preference,whereas anytime-anyplace computing does not.” (p118)

Recall that as a design philosophy, typology recognizes how creativity does better with themes and variations than with arbitrary innovation. It provides a framework for convention and invention to temper one another. Between conformity to a one-size-fits-all design and the chaos of infinite combinatorial possibility, there is a manageable range of recognizable situations. Design seldom benefits from infinite possibilities. It is more likely to be beneficial and appreciated when its variations occur on a few appropriate themes.” (pp118-119)

The difference between ubiquitous and situated computing appers vital. Ubiquitous computing (…) has mostly been a matter of pure mobility, with little regard for locally embedded systems. It has emphasized access to the same information everywhere. It has been geared toward connectivity 100 percent of the time for a few people, rather than providing information when useful for 100 percent of the people in a specific location. It has sought a one-size-fits-all solution for technological interoperability. By contrast, situated computing is based on the belief that such universality is neither attainable nor desirable. This approach questions total mobility, advocates local protocols, recognizes forms of tacit knowledge, and taps into more kinds of embodied predispositions.” (p142)

design must include some approach to appropriateness other than solely technological features and their performance specifications.” (p143)

Because technology affects so much of what we do, even who we think we are, its design involves judgment and appreciation.” (p148)

Now as computing becomes pervasive, the identity of these systems goes beyond the appearance of screens. New forms of ambient, haptic, and multiuser interfaces promote a shift from objects to experiences. Instead of emphasizing the visual identity of an object, under these circumstances we need to address the process of identifying with an experience.” (p157)

Subjectivity is inherent to usability. Differences in abilities, intentions, and exploration processes affect the successful use of technology at least as much as technical features. One way to represent this is with a “cognitive walkthrough,” which attempts to represent the sequence of assumptions, choices, and discoveries in the application of a technology. (…) this newer approach to user modeling has focused more on desire. Usability, identity, desire, and intent tend to relate.” (p160)

The majority of architects and designers still think it is their job to design the world from the outside, top-down. (…) To be fair, many younger designers feel free to set the stage for what is experienced. (…) People do like to be stimulated, to have things proposed to them. Designers are great at this. But the line between propose and impose is a thin one. We need a balance.” (p162)

Now physical computing arises from questioning the assumptions by which the graphical user interface became overblown. (…) Critical practice in interaction design works with a more open-ended and provocative story than problem solving in device engineering has typically done.” (p165)

As John Carroll has observed, “The worst misstep one can make in design is to solve the wrong problem.”” (p167)

The success of design is arrived at socially. Crudely, this suggests that market acceptance is the only criterion necessary. More to the point of critical practice, it suggests how design must help people understand a situation in a different way. (…) The very character of life and society has been transformed by planned artifice.” (p167)

By expanding the design of context-based information technology to reflect appreciation, experience, usability, and desire, more of us can contribute to the cultural assimilation of so much technical production. (…) Depending on choices we now face in design practices, interactive systems could similarly assume cultural meaning. In any case, they seem destined to surround us.” (p168)

We belong to several places and communities, partially by degree, and in ways that are mediated. (…) Places are a way of taking part in the world, for with a resonance unequaled by many other aspects of existence, they are both socially constructed and personally perceived.” (p171)

Interaction design must address how people move around, how they assimilate, and what kinds of local responses they encounter. As ever, design is for active, humane life; but without great precedent, now some contexts become active as well.” (p173)

Embodied activity grounds satisfying interaction design.” (p174)

When experience flows we get into place. Flow is of course an essential goal of interaction design, and fixity is an essential goal of architecture. Now the two join. To compliment the spaces of information with the contexts for getting into place, it helps to think in terms of ground.” (p191)

interactions establish value. Value emerges from interactions. This inquiring and examining has no end: what matters to individuals, societies, and markets never reaches a final equilibrium, but remains constantly in play.” (p194)

Aesthetic value must be culturally situated. It exists mainly at the convergence of qualified opinion. This may be what makes aesthetic value suspect to scientists: It is neither apparent nor consistent to everyone. Aesthetic value needs theory – and therefore critics – by which to deliberate its subjective expression and interpretation. These in turn benefit from being grounded in objective constructions, such as tonal scales in music. and genres, such as portraiture in painting.” (p196)

there must be a recognition of embodied pre-dispositions. From that it follows that cultural difference and local usage are much larger repositories of value than has been acknowledged to date. From this it follows that we need to find terms by which to measure such value.” (p206)

In the end, the design of technology cannot leave us as spectators and consumers, but must let us actively practice at something however humble.” (p207)

++”Paper In Screen” Prototyping: An Agile Technique To Anticipate The Mobile Experience [Bolchini, D. et al]

February 3, 2010

Taken from: “Paper In Screen” Prototyping: An Agile Technique To Anticipate The Mobile Experience.
BOLCHINI, D., PULIDO, D., and FAIOLA, A. (2009) “Paper In Screen” Prototyping: An Agile Technique To Anticipate The Mobile Experience. Interactions 16, 4 (July 2009), pp29-33.

For interaction designers to overcome a range of prototyping challenges, they first must recognize that each new and greater level of functionality in prototype development means more implementation time. This article introduces a hybrid method of prototyping that utilizes paper and mobile device technology that is both quick to create and agile to use in the early stages of design without the need to implement a fully operational high fidelity prototype.” (p29)

In discussing the relationship of user experience and design, Don Norman states that the “visceral” (or physical) level is the simplest and most primitive cognitive process [A]. With regard to handheld devices, visceral is all about look, feel, and sound, i.e., how a device, including the interface, looks and feels in the hand of the user. The iPhone is one of the greatest examples of the visceral experience. It was designed, in great part, for the visceral level of cognition—Apple designed for visual and physical impact. (…) Beyond the visceral experience, the “behavioral” level of cognition is about designing device interaction or behavior to reflect human behavior [A]. In other words, device design becomes intuitive in the way it complements one’s implicit assumptions about how it might work. Last, to design for the “reflective” level of cognition is to appeal to one’s aesthetic sensibilities, uniqueness, and cultural preferences [A]. From such a design perspective, people relate to and acquire a personal adherence to a device as part of their identity and self-expression.” (p31)
(A) Norman, D. Emotional Design: Why We Love (or Hate) Everyday Things. New York: Basic Books, 2003.

Understanding these three levels of cognition is extremely relevant, because emotional engagement at every level strongly influences human-interface interaction from a physical, aesthetic, and usability perspective.Moreover, if we need to take into account these emotion-centric factors early on in the life-cycle of device design, it is clear that paper prototyping cannot deliver the necessary insight into a full visceral and behavioral experience of the interface in the context of handling the physical device. In other words, if we only use paper separate from its actual relationship to the physical device, we may bypass important elements of the user’s emotional experience. As a consequence, inadequacies of this kind may lead to highly artificial (and ultimately irrelevant) evaluation results.” (p31)

The digital paper prototype is still provisional, malleable, thought provoking, and expressive, but at the same it enables the user to experience it within the real mobile device, with all its affordances for an interaction experience that is both tactile and visceral. And all this can be done at a very limited cost.” (p32)

Criticism As An Approach To Interface Aesthetics [Bertelsen, O.W.]

February 3, 2010

Taken from: Criticism As An Approach To Interface Aesthetics.
BERTELSEN, O. W. and POLD, S. (2004) Criticism As An Approach To Interface Aesthetics. In Proceedings of the Third Nordic Conference on Human-Computer interaction. Tampere, Finland, October 23 – 27, 2004. NordiCHI ’04, vol. 82. ACM, New York, pp 23-32.

today’s dominating perspectives on interactive artefacts focus almost only on technical and cognitive aspects, and consequently the field needs to take a cultural and aesthetic level of analysis into account in order to be able to address issues like design for unanticipated use or design of cultural interfaces. With the popularity of the PC and the web the interactive artefacts have spread from being efficient, functional tools at the workplace, to become a medium for cultural activity. Today, interactive artefacts are important media for producing, consuming and interacting with cultural data, e.g. on the web, or in computer games. Furthermore, it is also a cultural medium in its own terms, in the sense that interacting with interactive artefacts is an increasingly important cultural activity, (…) Interactive artefacts entered the cultural sphere long ago – this trend is accentuated by the current developments towards pervasive and ubiquitous computing.” (p23)

Apart from the media aesthetic branch of aesthetic theory, the aesthetics we propose is based on digital aesthetics. Theories of digital aesthetics have evolved within the field of aesthetic theory, especially since the PC was popularized, drawing on modern aesthetic theories from e.g. Barthes, McLuhan and Benjamin. Especially the developments of Benjamin focus on how digital art and aesthetics explore and develop a critical insight into the media and mediated perception, and how a new formal language is developing.” (p24)