Against experience design
The term “experience” is frequently used but little understood in HCI or design, including sub-fields of User Experience and Experience Design. I describe the results of a systematic survey of the HCI and design literature, and cross reference my findings with definitions of experience from relevant sources in philosophy and anthropology, to form a deep investigation of the term. The emerging framework for design and evaluation is structured around processes of transduction, transformation, and transactions. I adopt a critical approach which questions the long-held centrality of the human in HCI and design, in favour of a broader, critical perspective that positions humans within systems operating at multiple levels and timescales.
Introduction
I wrote this article while I was running the Information Experience Design programme at the Royal College of Art in London. I was motivated to research and write it based on two converging trends I observed in relation to experience — an overused term, pervasive in human-computer interaction (HCI) and design, and increasingly a focal point of both research and practice.
The first trend is well articulated by Hornbaek and Hertzum [32]. Based on an extensive review they conducted, they conclude that the “experiential component in HCI is not yet well understood.” Vliet and Mulder [59] similarly propose that HCI research should be broadened and infused with research on experience from philosophy and other fields.
The second trend is an increasingly critical approach to technology, both within and beyond the technology and design communities, e.g. [27,48]. Gray et al [28] discuss “dark patterns” of commercial exploitation in User Experience (UX). Bødker [4] observes that during what she identifies as “Third Wave” HCI, “it is important not to just ‘dump’ technology on people — which, unfortunately, is what has happened in this period.” As a consequence, Kaptelinin draws attention to the ethical responsibilities of the HCI community, noting that digital technologies increasingly “redefine our social and material world and become a significant factor shaping our identities” [36]. Other recent work (e.g, [62]) has begun to regard technology to just another “material for designing experiences.” Boehner et al [5], stressing that “systems codify but their outputs become messy in human experience,” focus on “designing for the negative space around objects.” Wright et al [67], among others, suggest that the concept of the “user” may be unhelpful in HCI. Wright and McCarthy [66] note, however, that “ethical and political ideals are seldom articulated within the HCI research community,” and Gray et al [28] point out, “the vocabulary for describing and assessing criticality in practice is currently lacking.”
Based on these trends, I conducted a systematic review of relevant HCI and design literature, and identified broad disparity in the use and understanding of the term “experience.” I discuss this in relation to definitions of experience from foundational sources in anthropology and philosophy. My aim is to begin to build a shared understanding in the design and HCI communities around the concept of experience. According to Hornbaek and Hertzum [32], reaching consensus about this concept “is a key to working systematically toward understanding the experiential component.”
The contributions of this article are therefore: 1) an expanded framework of experience as a basic unit of analysis, which can used for design or evaluation, grounded in theoretical and practical findings; 2) I interrogate techno-centric and commercial approaches to UX and experience design (XD), identifying ethical implications for design and HCI; and 3) I set out directions for further research and practice focused on experience.
My approach
First, I first wanted to know how experience has been defined in HCI, including sub-fields of UX and XD. Second, I wondered how definitions in these fields compare with key definitions from anthropology and philosophy. Consequently, I sought ways experience or experiences might be studied and designed. Finally, I wanted to find specific characteristics that might form a common framework grounded in experience to inform research and practice.
To investigate how experience has been defined and used in HCI and design, I did a systematic review of relevant literature in these fields. (I detail the specific methods I used in an appendix at the end of this article.) I then cross-referenced the definitions I found with those arising from philosophy and anthropology. I chose these as academic fields concerned with human experience, and which I regularly draw upon in my own teaching and research. Pragmatist philosophy, particularly that of John Dewey, has particularly informed HCI since the turn of the 21st century [19, 43, 53] and necessarily takes some prominence in my study as well. But I identify other relevant and more recent philosophical strands. I did not include psychology; while it has had a prominent role in the development of HCI, the balance has been shifting as HCI has moved away from solely cognitive approaches and towards sociocultural [37] and critical [15] ones. I do however bring in a few references from psychology, neuroscience and education to support some of my claims. I analysed the resulting definitions to identify mechanisms and characteristics of experience, then compared and combined them to build a framework.
Defining experience
The term “experience” is “simultaneously rich and elusive” according to McCarthy and Wright [43], alternately used as a noun or verb, with nuances even within this. As a verb, individuals experience particular internal states or external phenomena, while as a noun we speak of “the experience of” such states or phenomena. We may have 20 years of experience, or 20 years of accumulated experiences; we speak of individual experience and collective experience. Experience as a verb is broadly viewed as subjective, while as a noun referring to a bounded phenomenon or situation that many presume can be empirically studied and designed. As Vliet and Mulder [59] point out, most theories and models of experience are descriptive and not explanatory, and I discuss below the limitations of empirical approaches.
Looking across all the literature I surveyed, experience is often counterposed against interpretation, with the former framed as transcending language and the latter reliant upon it. McCarthy and Wright, for example, define experience concisely as “life as lived, not just as theorized” [43]. Karana et al. [38] describe it epistemologically as “a way to know the world and to enrich knowledge of it.” Boehner et al [5] describe it in terms of “the ineffable” as that which cannot be easily put into words. In summary, we are all experts in experience, but it can be difficult to articulate the concept itself with words. Thus my first observation is that language is limited, both in conveying experience and in research methods that rely on written or spoken interpretations.
Experience in HCI and design
Experience has been a common theme of HCI conferences and workshops]; it defines the sub- fields of UX and XD, the latter increasingly supplanting the former with its broader focus and the shift away from the term “user.” In my reading of the literature, I would broadly characterise UX as more focused on evaluation, arising from usability studies; and XD as more focused on designing bounded experiences, usually for commercial gain. UX has, however, employed broad definitions of usability which are similar to what is now labelled as experience [59].
Most approaches tend to be grounded in cognitive assumptions. As Wright and McCarthy point out, “The cognitivist perspective was particularly valuable [in HCI] because it offered a language and a set of values that both psychologists and computer scientists could understand and commit to.” [66]
Experience in HCI is necessarily tied to both humans and technology, with either technology or experience seen as mediating the other [58]. McCarthy and Wright shifted their focus from “technology as experience” [43] to “experience-centered design” [66] to emphasise a more humanist approach, while remaining focused on technology.
There are many terms related to experience in the HCI literature. Hornbaek and Herzum [32] identify “aesthetics,” “affect,” “appeal,” “emotion,” “engagement,” “enjoyment,” “flow,” “fun,” and “hedonic” just within UX research, which they find overwhelmingly focused on perceived enjoyment and positive emotions. During “Second Wave” HCI, focus shifted from function to pleasure and aesthetics. Geven et al point out that this lacks “a more profound understanding of underlying processes, of ‘experiences’ as such,” and suggest further exploration into experience theories [25].
In design as well, success is often defined in terms of “how people positively experience and react to the materials chosen by designers.” [38] “Everything depends on the quality of the experience which is had” according to Shedroff [53].
“Addressing the human experience,” according to Gajendar [23], “has become the central task for designers today in the Deweyan sense.” With regard to experience, all roads seem to lead to John Dewey, whose pragmatist view on experience as both noun and verb is detailed in [11,12,13]. This includes McCarthy and Wright, who rely heavily on Dewey in their influential book Technology as Experience [43].
Phenomenological approaches in HCI are closely related (eg [14]), being similarly grounded in philosophy — particularly that of Merleau-Ponty [46]. Some have brought in other philosophical traditions, most recently Kaptelinin [36], who discusses existentialism in relation to human experience broadly.
In most of of the papers in my initial search (98 of 105), while the term “experience” appeared in the title, it was not explained or defined in the text. Therefore, in the next sections I analyse the term, comparing its use in HCI and design with its definition in relevant philosophical and anthropological texts, in order to inform a useful conceptual framework for design and evaluation. In the next section I discuss experience as a verb (to experience); then I discuss the notion of time (experiencing); this leads into an analysis of experience as noun (experiences).
To experience
In defining experience as life lived, not theorised, McCarthy and Wright [43] observe that “we can never step out of experience and look at it in a detached way.” It is direct, first hand and first person. As such, it is by nature subjective and (importantly) controlled by the individual. Sumi et al [55] for example describe the use of first-person video in HCI as “experiential,” though I would characterise this as mediated experience.
The subject
Even without technology, experience can be said to be mediated by individual bodies — it is embodied. In philosophy, Merleau-Ponty is best known for this view, [46] regarding the body as the condition and context of being in the world, thereby structuring our experience and placing limitations and opportunities on what we can experience [44] Dourish [14] introduced this view into HCI, yet the body is still lacking from much UX research, which takes experience to be purely mental perception, satisfaction or enjoyment of software applications [32].
Experience design seemingly pays more attention to embodied experience. For example, Farnham and Newbery [17] mention the Feldenkrais method of bodily awareness. However, this is used only as an analogy for changing business practices.
The anthropologist Edward Bruner observes that experience is not equivalent to behavior:
“The latter implies an outside observer describing someone else’s actions…experience is more personal, as it refers to an active self, to a human being who not only engages in but shapes action. We can have an experience but we cannot have a behavior; we describe the behavior of others but we characterize our own experience.” [8]
Perception is also not experience, according to the philosopher Alva Noë. He calls experience “bringing the world into focus” — not just something that happens inside us as a result of being affected by the world around us, because “we see much more than projects to the eyes. We experience what is hidden or occluded (the tomato’s back, for example); we experience the nature of things (what they are — telephones, say, or other people); we perceive emotion and meaning (the intensity of a person’s concentration; what she is saying).” [47]
As something individual and subjective, experience is therefore “something that one fabricates oneself, that doesn’t exist before and will exist afterward.” [21] In HCI as well, the experiencing individual is seen as actively creating their own experience [25].
Objects
The experiencing body is situated in the world — in particular social and physical contexts. The scope of experience is thus a matter of what is available to us, according to Noë, in a particular context, and on what we can do. “What we can do,” he says, “depends in turn on understanding, know-how, but also on tools and technology…, and on where we find ourselves and what environmental or social resources are available to us.” [47]
Both HCI and design are mainly focused on the tools and technology that Noë refers to. UX models “are in essence about the experience of interactive products, the consequences of those experiences, and the ways experiences and consequences are connected.” [32]
In design, Karana et al [38], focus more on materials than tools, referring to “the mix of sensory (or aesthetic) appreciations, meanings, feelings, and thoughts that we have toward — or that are triggered by — a material, at any certain time and place.” They then suggest, “Understanding sensory modalities is therefore a critical step if we are to design for materials experiences.”
Contexts
Social and environmental resources, and times and places in which materials are used, connect to Gibson’s [26] environmental psychology, particularly his theory of affordances which has had broad influence in HCI. While many refer to affordances in a cursory way, however, few reference Gibson’s grounding in evolution and biology (only one [59] in my sample.) About half of the UX studies reviewed by Hornbaek and Herzum [32] did not account for context of use.
Korhonen et al [42], operating within UX, state, “A user’s interaction with a product in certain contexts defines the user experience,” and refer to experience “that results from the interaction with a product with relation to the triggering context element.” By “triggering context element” they identified six contextual factors: environment, personal, task, social, spatio-temporal, terminal, service, access network.
Suchman [54] equates “situations” with the physical context. According to Farnham and Newbery [17], “Perception is strongly affected by the situation or context.” Dourish [14] discusses social and physical contexts, as do Forlizzi and Battarbee [19] in relation to “co-experience.” Some of my previous research [60] studied the social and physical as well as the “personal context” which was seen to be linked to individual experience.
The social context (or broader socio-cultural context [16]) includes other individuals as resources for, and participants in, experience. Gibson discusses “social affordances” in this regard [26]. According to Geven et al. [25], “Communication with others is also part of an experience.” Co-experience, for Forlizzi and Battarbee [19], is a special category of experiences that are created and shared between people. Hornbaek and Herzum [32], however, note that accounting for social aspects of experience is underdeveloped. Indeed, I do not address this in the current article, instead focusing on individual experience, but I propose one direction for further research below.
Transduction
Regardless of how context is defined, McCarthy and Wright point out that experience produces enduring changes, not only to the subject: “The world is changed by the outcome on the world of the total action and also by the changes brought about in the experiencer, whose sense of self may be transformed, and whose perspective and attitudes are likely to have changed.” [43]
If experience is mediated by our bodies and senses in relation to other things in particular contexts, what Noë calls “bringing the world into focus” necessarily involves some process of mental translation.
The philosopher William James provides a view I find useful and under-appreciated. He suggests that it doesn’t make sense to think of spaces, bodies or things; there is only “pure experience,” of which everything else is composed — one experience is the knower, the other the reality known. The individual exists in the world, but the world encountered also exists inside the mind of the individual. Thus experience consists of both thing and thought, percept and concept, content and consciousness, experiences that act (have consequences) versus those that do not (thoughts). Things both are and are known, and “truth” is both subjective and objective. [33]
This dual truth is complicated by how exactly the process of translation is undertaken. While noting again the limitations of language, I prefer the term “transduction,” for its shared meaning across scientific fields. We can separate it into interpretation and understanding. According to Shusterman, interpretation relies on words, whereas understanding does not: “a shudder, a tingle, may be enough to indicate that one has understood. Some of the things we experience and understand are never captured by language, not only because their particular feel defies adequate linguistic expression, but because we are not even aware of them as ‘things’ to describe.” [43]
Dewey similarly believed in the inseparability of emotion and behavior [66], and this is now supported by neuroscience: “How we feel affects the way we interpret internal and external signals, and our interpretations affect the way we feel” [41].
The “why” of experience is strongly related to critical understanding, according to Noë, again related to emotion — how we feel about what we encounter. “Perceiving is an activity of securing access to the world by cultivating the right critical stance, that is, by cultivating the right understanding.” [47] In design, according to Friedman, “It is not experience, but our interpretation and understanding of experience that leads to knowledge. Knowledge emerges from critical inquiry.” [22]
Experience is therefore more than the sum of its parts [25], an “irreducible totality of people acting, sensing, thinking, feeling, and making meaning in a setting,” according to McCarthy and Wright [43]. For this reason they view it in terms of interwoven threads — sensual, emotional, compositional, and spatio-temporal. Geven et al [25] refer to a “Heisenberg principle in user experience research,” that we cannot assess all these factors at the same time, and by measuring one, we influence the others. “Hence, we remain uncertain about the total user experience.”
Accessing experience
Several researchers have tried to untangle the threads to isolate, for example, emotions (e.g. [6]). Dewey observes, however, that emotions are not things but qualities of experience which change over time, and thus capturing discrete instances of emotional states has limited use [10].
Various methods are used in attempts to capture experience more broadly. Video recording is common, including in my work [61]. Sumi et al [55] use “experiential data” (multiple perspective video, sensor-enabled objects), combined with metacognitive verbalization in which participants articulate what they are thinking and feeling in situ. Since narrative is seen as making action intelligible [44], talk-aloud protocols are common, including in my research [60] According to Forlizzi and Ford [20], such self-talk or self-narration is intended to access the constant stream of consciousness in relation to “the purest form of reference,” echoing James’ definition of “pure experience” above.
Following from this, meaning making can be defined as an internal transduction of relations between a subject’s personal experience with an other’s [60]. The only way we can access such subjective meanings, according to Geven et al, is through some means of communication: “we will always have to ask users what something means to them…. to find out what interpretation is given to experiences by a user” [25].
Most studies of experience in HCI have thus engaged participants in reflective storytelling, questionnaires or diaries completed after experiencing a product or system. For example, “experience reports” were used by Korhonen et al. [42] to assess personal feelings, values and interests, after use, then key words were cross-referenced to 13 “pleasure categories”. Hassenzahl [30] also studied hedonic, as well as pragmatic, attributes using a questionnaire.
Geven et al [25] distinguish between reporting an event and telling a story, the former providing a simple recount of the situation, and the latter allowing participants to “really re-live the experience,” by recounting related emotions felt during the experience. Farnham and Newbery point out, however, that feelings can be very different when they are experienced and recounted afterward. “We tend to evaluate based on what happened during the latter parts of an experience and remember the most intense levels of experience — we can have a reasonably good time, but if it ended abruptly or badly, that’s all we take away” [17].
In “experience sampling,” participants are prompted to complete short surveys immediately after performing behaviors of interest. “Experience diaries” [5] are a “light-weight, communal, aesthetically focused snapshot,” intended to go beyond evaluation to part of the overall experience. Wright et al. [67] describe a designer making a book of images as a more tacit, emotional and empathic means of relating to participants’ responses. Cultural probes [24, 25] involve participants in responding to a set of designed, object-based stimuli.
Research by some of my PhD students and colleagues has involved participants in making abstract drawings to represent emotions [69], and engaged participants in constructing two- and three-dimensional representations of digital experiences [18]. These were shown to involve an “unflatenning” of experience from digital screens and interfaces, a reflective distancing from their immediate effects, and the generation of new metaphors that were useful in accessing understanding nonverbally. Similarly, having participants construct physical manifestations of immersive spatio-temporal experiences has shown longer-term effects on memory and working practices [51].
In order to further my aim of constructing a useful working definition of experience, I am therefore drawn to the term “expression,” following the anthropologist Victor Turner [57], which encompasses externalisations of experience which can take written, constructed, performed or other forms. McCarthy and Wright point out that engaging in expression (even completing a survey) itself constitutes experience — “the experience of describing, not the experience being described.” [43] Expressions, Turner [57] says, in turn must be enacted to be experienced. But they are usually bounded and so easier to study than the continuous flow of experience, though losing the vitality of the latter.
Experiencing
As a continuous flow, pure experience for James as a basic unit of understanding is always constructed in the present. The experience of another thing, as an external objective reality, may have existed in the past, but may not have existed in an individual’s mind until now. Similarly for the contemporary philosopher Byung-Chul Han [29], “Truth itself is a temporal phenomenon. It is a reflection of the lasting, eternal present.” Thus, experience is connected also to the notion of being present, as a constantly shifting condition.
The philosopher Wilhelm Dilthey, James’ contemporary, proposed that experience is exactly the link between past and present [57]. For Dewey also, all experience is both continuous and cumulative [53], resolved as lived understandings and interpretations based on past experience, contextual knowledge in the present, and anticipated reactions [44]. Similarly, from an anthropological perspective, “with every experience in the present we have one eye on the past and the other on the future,” according to Bruner [8]. But he points to a paradox:
“Although life is a flow, we can never experience that flow directly because every observed moment is a remembered moment. Temporal succession cannot be experienced as such because the very observation of time fixes our attention and interrupts the flow of experience, leading to periods of reflexivity when the mind becomes ‘conscious of itself.’” [8]
This puts into question HCI methods such as experience sampling or talk-aloud protocols, which interrupt the flow by asking participants to engage in written or spoken expression. By contrast, “experience recording” technologies such as video and various types of sensing [55, 65, 49] may hold promise, though awareness of them by participants also tends to affect the experience itself, also raising ethical issues which are beyond the scope of this article.
Vliet and Mulder [59] developed an analytical framework that takes time into account, encompassing memories, the present, and expectations, with two other axes of mindset (reflective or not) and intensity of emotion. Their model includes relations between designers (who set expectations and can control intensity to some degree), participants, and artefacts, in a manner similar to cultural probes [24].
The notion of experience as a continuous flow implies linearity and direction, connecting past, present, and future. “As we can only enter the world in the middle, in the present,” according to Bruner, “then stories serve as meaning-generating interpretive devices which frame the present within a hypothetical past and an anticipated future.” [8] Experience itself can be considered a story, as Forlizzi and Ford[19] discuss. But other research [50] has found that individuals experience time itself in different ways, based on chronobiology, and this can inform design decisions. Experience should therefore be regarded as temporally variable and nonlinear, encompassing multiple flows and narratives which may intersect in different ways.
The transformation of experience over time can be called “learning.” Dewey refers explicitly to “experiential learning,” which involves direct, bodily engagement with phenomena [13]. This relates to another definition of experience: life experience.
Experiences
In contrast to the wide timespan of experience is what Han [29] calls “the punctual, timeless experienced event.” Forlizzi and Battarbee [19] follow Dewey: “An experience is a stream that has coalesced into something with a beginning, middle and end, which can be named, talked about and evaluated.” A common question in design and HCI is articulated by Geven: “But what is such an experience (theoretically), and how can we measure such experience (methodologically)?” [25]
Common across the literature is that experiences are consciously recognised as such by individuals, with a clearly defined beginning and end, signalled by “internal experiences of time” according to the psychologist Heider [34]. Related terms in the HCI literature include “activities,” defined for example in activity theory as goal-oriented systems consisting, minimally, of subject, object and motive whereby the subject transforms the object into an outcome [37]. Other terms include “encounters” [23], “episodes” [32] or “experience episodes” [32]. Vliet and Mulder [59] use an analogy of “snapshots” versus “movies.”
The most common related term is “situations,” associated in HCI mainly with Suchman [54]. Geven et al [25] use “experience” and “situation” interchangeably, stating, “situations are set off as self-contained wholes by virtue of an immediate ‘quality’ that pervades each situation.” But broader use of this term is more vague. Sartre refers to “one’s situation” as a politico-economic position — one’s lot in life. Goffman defines it as “an environment of communication possibilities.” The radical group of artists and intellectuals known as Situationists defined it as “integrated ensemble of behavior in time.”
Emotion
An experience is commonly defined across design, HCI and philosophy as having particular, often emotional or ineffable [5] qualities that set it apart from the stream of experience, of everyday “situations” such as work [TAM]. Automatic actions are not considered experiences [10] — a difference in quality, not as in activity theory’s hierarchical distinction between activities, actions and operations. Dewey characterises experiences in their uniqueness, unity of emotions, and wholeness [43], as being consummate examples of a type [12].
Vliet and Mulder observe that the role of emotions in UX research “is seen as a consequence or as an antecedent in relation to the product” [59]. Again, UX research has overwhelmingly described positive emotions in relation to technology acceptance and use. [32]
Aesthetics
Aesthetics is a term closely associated with experience in the HCI literature [Hornbaek], but as Karana [38] note, aesthetics, meanings, and emotions are not easily isolated. Aesthetics has a long history in philosophy, today most commonly associated with art, which Dewey defined as “refined and intensified forms of experience” [12]. This holds not only for those who experience art but those who create it, for whom making is an experience as well: “the artist embodies himself in the attitude of the perceived while he works” [12]. As my own work often blurs the distinction between art and design, I would substitute for ”artist” the word “designer”, following Vliet and Mulder [59] who studied designers’ expectations against users’ experiences and reflections. According to Karana [38], “designers and users together are the principal stakeholders in the loop that joins intended and realized materials experiences.”
According to Vliet and Mulder, “One view on aesthetic experience concentrates on the properties of the object that lead to a direct sense of beauty that cannot be ‘escaped’ (like the ‘affordances’ in perception)” [59]. According to Dewey, “That which distinguishes an experience as aesthetic is a conversion of resistances and tensions, of excitations that in themselves are temptations to diversion, into a movement toward an inclusive and fulfilling close.” [12]
Intensity
Turner describes “moments when life is lived most intensely” as aesthetic experiences. An emotional response is the conscious sign of a break, whether actual or impending, according to Dewey [57]. Unpredictability is therefore closely related to learning, or transformation. “Any experience that does not violate expectation,” according to the philosopher Hegel, “is not worthy of the name experience.” Such experiences are both formative and transformative, according to Turner [57].
I again note that experience in both HCI and design has most often been associated with positive emotions; Vliet and Mulder [59] include “intensity” as one of the axes in their product experience evaluation model. According to Dewey, “struggle and conflict may be themselves enjoyed, although they are painful, when they are experienced as means of developing an experience”. [10]
Rhythm
Regardless of intensity, “an experience is something that one comes out of transformed,” says Foucault [21]. According to Dewey, “Experiencing like breathing is a rhythm of intaking and outgivings,” [10] with an internal, dynamic structure consisting of cumulation, conservation, tension, and anticipation. McCarthy and Wright [43] translate these into specific aspects of technology, including aesthetic implications, for evaluation or design.
Dewey similarly uses the concept of “flow” to describe the internal dynamics of an experience: “In an experience, flow is from something to something…. there are no holes, mechanical junctions, and dead centers when we have an experience. There are pauses, places of rest, but they punctuate and define the quality of movement.” [10]
Csikszentmihalyi uses “flow” to describe states of internal motivation, though his focus is not on aesthetic experiences but optimal task performance. Csikszentmihalyi and Hermanson, however, propose a model for the design of “flow experience,” based on intrinsic motivation, which includes sensory, emotional and intellectual means of involvement; in alignment with the previous discussion of the unexpected, their model suggests that a balance of expectation and novelty characterise good flow experience. In HCI, Bian et al [] study flow experience in virtual environments, in relation to task performance. Hornbaek [32] review HCI studies of flow. Bruhlmann et al [] study intrinsic motivation in relation to user experience during interaction, but do not refer to flow.
Rhythm or flow can also take place between as well as within experiences, as a continuity between aesthetic and prosaic experiences. For Dewey, “every experience takes up something from those which have gone before and modifies in some way the quality of those which come after” [12]. According to James, “Life is in the transitions,” and Han [29] contends, “Today’s experience is characterised by the fact that it is very poor in transitions.” This has been addressed in sub-fields such as transition design [23], and in HCI in debates between “seamless” and “seamful” design.
Vliet and Mulder [59] developed and tested an “Experience Expectation Instrument” and “Experience Observation Instrument”. They describe “a temporal rim around an experience” with pre- and post-experience stages in which expectations are brought into the situation, and memories come out. Their research was seen to validate these models, but they questioned their own use of surveys as a method for evaluating experiences.
Longer-term research might look at experience across experiences. As Bødker [4] observes, “one of the questions that remains is whether we have become better at studying these multiple, experience-based use situations, methodically, empirically, or conceptually.” This review, in highlighting the inherent subjectivity and complexity of experience, casts doubt on empirical study, but I hope to contribute to a conceptual framework for the study and design of experiences.
Transactions
In addition to the challenges of studying experiences, several researchers in design and HCI question whether we can design them, suggesting that “experience design” is an oxymoron. Forlizzi and Battarbee [19] question whether we can even script experience, “or simply be content to facilitate it or keep from hindering it.” They believe we can only “craft an experience, we can only design situations, or levers that people can interact with, rather than neatly predicted outcomes.” According to Wright et al [67], “What designers can do is provide resources through which users structure their experiences.” Karana et al. [38] also state, “No matter how much we design for all users, an idiosyncratic experience is created; the experience is shaped by the meaning that the user gives to an interaction which varies for each user.” Gajendar [23] similarly asserts, “We don’t actually design experiences. We design only the contexts, interfaces and artifacts that might lead to a positive experience.”
It is outside my scope to define “design,” but we could similarly view design as a verb instead of a noun. This shifts focus from Experience Design (or any type of design) as a monolithic field to a mere process, as it is used in all the references in the previous paragraph. Shifting to a processual view exposes practices which bridge between research and design, such as “experience prototyping” which engages designers in enacting experiences as part of the design process, creating and testing low- to progressively higher-resolution prototypes. Research-through-design similarly engages designers in creating things not as products but for research purposes, which do not necessarily need to be tested with users [44].
Karana et al. [38] describe designers and users working together as “stakeholders in a loop that joins intended and realized materials experiences.” Similar to Dewey’s discussion of art as an experience that links artist and viewer, aesthetics is located in craft [67][69].
Viewed over time, the relations between designer, object or system and user can be characterised as transactions. For McCarthy and Wright [43], drawing from Dewey, “‘experience’ refers to transactions between us and the objects and events that make up the world in which we act.” Kolb [] similarly prefers the term “transaction,” “because the connotation of interaction is somehow too mechanical, involving unchanging separate entities that become intertwined but retain their separate identities.” For him, transaction implies a more fluid relationship in which both sides are changed.
Value
If “interaction” is too mechanical, might “transaction” be too monetary? What exactly is exchanged in such transactions? To address this, I interrogate a related term that emerges in the literature — value.
In Gajendar’s [23] “integrative aesthetic experience” model, “aesthetic implies a complete and total sense of human value”. Alben [2] states that successful experiences must be “valuable” to users. Sumi et al [55] discuss how technology “contributes to value creation in human consciousness… such as reflecting and weaving our own experiences and memories.” Wright and McCarthy [66] situate themselves in the humanist tradition in HCI, “which focuses on enriching aspects of our humanity through attending to values of agency, democracy, equality, and choice.” They however note, “Even in the new contexts of experience-centered design, the humanist impulse and agenda can be hijacked to serve brand identity and product attractiveness and desirability, instead of enhancing the lives of people who buy the products.” Vliet and Mulder [59] discuss specific ways some of the perceptual and emotional qualities of experience earlier can be manipulated.
Economy
The “experience economy,” what Gajendar [] calls a “now-empty phrase,” was coined by Pine and Gilmore [45]. As discussed by Forlizzi and Battarbee [19], they propose five principles for designing experiences: “theme the experience, fulfill it in all the details, harmonize the impression with positive cues, eliminate negative cues, and mix in memorabilia.” This parallels the emphasis on positive emotions studied in UX, discussed earlier; and as Forlizzi and Battarbee note, is situated in entertainment and customer service. According to Gray [28], this is “where user value is supplanted in favor of shareholder value.”
Experience design, as conceived in the commercial sector, is thus not about product or user experience, but experience instead becomes the product, associated with an “aestheticized lifestyle” [68]. Jones et al. [35], writing from an artistic, not commercial, perspective, ask, “If we can only know reality through our experience, how should we understand the production of experience itself?” More broadly, the artist Tino Seghal [52] refers to a shift from an “experiencing society” in the 1990s (when “the experience economy” was conceived) to an “experience society” today in which experiences are valued, produced, consumed and studied.
Viewed through the lens of value(s), however, this is not a simple case of commercial exploitation, and I recognise that both I and my readers live and work within a capitalist system. Writing some 20 years before “experience economy” was coined, Abrahams [1] highlighted the Americanness of the term “experience,” pointing out the American roots of pragmatist philosophy — chiefly Dewey and James. Experience design, as described in Farnham and Newbery’s [17] book of the same name, is defined simply as designing systems to deliver value to customers. This involves an asymmetry in value and information, they say, in which one side has more information or intangible value, prompting a transaction. Intangible value includes subjective qualities such as emotional or psychological needs. But the authors do not detail specific techniques for generating emotional responses:
“There are times when using such techniques is very smart… But we believe that if there isn’t sufficient value to begin with, these techniques by themselves do not create long-term engagement… Many designers and buyers of design services look at these techniques as being what experience design is all about, but we strongly suggest that this is a myopic view, and that approaches that target emotional responses be used to increase real value, not as a substitute for it.” [17]
The authors point out that new technologies enable the nature of value to change, and a fundamental aspect of experience design for them is time, echoing what I have discussed — “how we interpret what is happening at particular times in terms of meanings and feelings, and what these imply with regard to future actions and experiences” [17].
Systems of experience
In 1996, Winograd and Flores [64] proposed that “designing for the full range of human experience may be the theme of the next generation of discourse of software design”. A decade later, emotional design and experience design [53] were entrenched in both design and HCI, but Vliet and Mulder voiced a growing concern in the community: “We find it equally interesting and important for the development of ‘user’ experience to gain insight in the appraisal of the user of that interaction as well as gain insight in the appraisal of the product (e.g. computer) of that interaction” [59]
Moving ahead another decade or so to today, humanist approaches to HCI have thus come to the fore, as detailed in [4] and [66]. But Vleit and Mulder’s call for equal attention to be paid to users as well as products implies placing them on an equal level. Based on my review of the relevant HCI, design, anthropology and philosophy literature, I wonder if it might now be appropriate to reverse these roles, and more broadly to view experience from the perspective of particular entities, whether human or not. (How) might humans be regarded as products, and agency be assigned to nonhuman things? If experience is beyond design, does it apply to nonhumans? Is design a solely human endeavour? Careful readers may have noticed that many of my discussions of experience in this article have referred to “subjects,” “participants” or “individuals” and not “people.” Similarly, I chose the term “transduction” for its use in biology as well as machine learning and engineering.
A systems approach situates agency in components which are themselves systems, in nested, communicative relations. Bratton [7] points to rapid advances in machine learning combined with shrinking hardware to suggest that a human-centred approach is insufficient for understanding the increasingly global, distributed nature of computing systems. Conversely, individual humans as systems are composed of smaller systems, and HCI and design both still largely ignore the role of the body. Moreover, every thing, in systems theory, is temporary: “We are but whirlpools in a river of ever-flowing water,” according to Weiner [63]. “We are not stuff that abides, but patterns that perpetuate themselves.” This matches with William James’ conceptualisation of “pure experience” as all there is, and contemporary philosopher Federico Campagna’s [9] view that our reliance on language has resulted in “things” being replaced by “positions.” This is likely to be a controversial provocation to the HCI community, effectively reversing human-centred approaches.
Conclusion
Expressing our work and ideas to other humans will usually involve using language (such as in this article), but understanding, in Shusterman’s definition, need not. I have reported results from a survey of HCI and design literature on experience, comparing resulting definitions with those from phliosophy and anthropology, to identify mechanisms of transduction (comprised of critical understanding and expression), transformation (temporality, rhythm, aesthetics), and transactions (of value). The emerging framework suggests a systems perspective which shifts away from the current human-centred paradigm. Taylor [56] has already questioned the “I” in HCI; might it be time to question the “H” and the “C” as well?
According to Kaptelinin [36], “HCI appears to be experiencing some sort of an identity crisis.” To address this, he suggests that “a certain overlap between HCI and existing fields is inevitable and probably even desirable.” He explores existential philosophy, while I propose extending this to broader strands of philosophy around experience, a term I have found to be simultaneously too variable and too restricted in the design and HCI literature.
If experience is beyond design, then understanding the conditions of its production and consumption (what Vliet and Mulder call the experience of experience) in terms of transduction, transactions and transformation might advance the knowledge of designers, technologists and businesspeople, no matter what the context of particular experiences. “if positioning experience in the context of the work to be done to develop a democratic culture does nothing else,” according to McCarthy and Wright [43] “it takes it out of the slippery world of packaged and marketed experiences and places it firmly in the ordinary, everyday worlds of people working at life.”
APPENDIX
Methods
To investigate how experience has been defined and used in HCI and design, I did a systematic review of relevant literature in these fields. This began with a search for the term “experience” in titles of publications in HCI (TOCHI, Proceedings of the CHI conference, Interactions magazine), broadening the search somewhat to include relevant results from other ACM publications that met my criteria (see below). To investigate experience in the design literature, I undertook similar searches of Design Issues and She ji: The Journal of Design, Economics, and Innovation.
My selection criteria were that the publication had to focus explicitly on the study and/or design of experience or experiences. I proceeded from an initial assumption that “experience” would thus be in the title. I restricted my initial search to the past decade, to limit my sample to a manageable size, and on the assumption that this would best represent the latest thinking. This period roughly aligns with Third Wave HCI as identified by Bødker [4], in which experience has taken greater prominence over usability.
Following Hornbaek and Herzum [32], I then used the method of backchaining citations to identify key foundational references, including books as well as journal articles and conference papers. Some of these date back to the early 2000s, and I included them in my sample. This yielded 105 results.
Of these, I determined from the titles and, where necessary, abstracts, whether publications focused primarily on experience as an analytical unit or as part of a conceptual framework in research, as a primary framing or outcome in design. As the term is so pervasive, and used in different ways, I avoided common usage, such as when “experience” is applied to other phenomena, unless framed as a prominent focus. I also ignored generic usage, as in “users had a positive experience.” This yielded 26 usable results. I then read these publications to analyse whether and how each defines experience.
I next cross-referenced the definitions with those arising from philosophy and anthropology. I chose these as academic fields concerned with human experience, and which I regularly draw upon in my own research. Pragmatist philosophy, particularly that of John Dewey, has particularly informed HCI since the turn of the century [19, 43, 53] and necessarily takes some prominence in this study as well. But I identify other relevant and more recent philosophical strands. I did not include psychology; while it has had a prominent role in the development of HCI, the balance has been shifting as HCI has moved away from solely cognitive approaches to the sociocultural [37] and critical [15]. I do however bring in a few references from psychology, neuroscience and education to support some of my claims.
I analysed the resulting definitions to identify such mechanisms and characteristics, then compared and combined them to build a framework.
Limitations
Experience is such a broad topic, I necessarily had to set some bounds to my study. While my sampling strategy for the HCI and design literature was systematic, I was much more subjective in selecting other sources. They are thus restricted to those I identified through backchaining from the HCI and design literature, and from my own knowledge as to relevance. This study therefore represents one “reading” of experience, and is intended to prompt a conversation within the design and HCI communities. More specifically, I read the HCI and design literature on experience through the other fields, particularly through a philosophical lens, as a “diffractive reading” following Karen Barad [3].
As noted above, I also take an explicitly critical perspective. Accordingly, my research is much more qualitative and subjective than, for example, Hornbaek and Herzum [32], and the bulk of this paper is devoted to an extended discussion of the topic. As such, I do not separate out quantitative findings from literature, but interweave these within the discussion.
REFERENCES
[1] Roger D. Abrahams. 1986. Ordinary and Extraordinary Experience. In Turner and Bruner (Eds.) The Anthropology of Experience. Univ. of Illinois, Chicago.
[2] Lauralee Alben. 1996. Qualities of Experience: Defining the criteria for effective interaction design. Interactions 3(3), 11–15.
[3] Karen Barad. 2007. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Duke Univ. Press.
[4] Susanne Bødker. 2015. Third-wave HCI, 10 years later — -participation and sharing. Interactions 22(5), 24–31.
[5] Kirsten Boehner, Phoebe Sengers, Simeon Warner. 2008. Interfaces with the ineffable: Meeting aesthetic experience on its own terms. ACM Transactions on Computer-Human Interaction (TOCHI) 15(3) Article 12.
[6] Kirsten Boehner, Rogério DePaula, Paul Dourish and Phoebe Sengers. How emotion is made and measured. International Journal of Human-Computer Studies 65(4), 275–291.
[7] Benjamin H. Bratton. 2018. Quote Unquote Design: Landscape-Scale AI and the Question of Agency. Keynote speech at Beyond Change: Questioning the role of design in times of global transformations Swiss Design Network Research Summit March 8–10, 2018.
[8] Edward M. Bruner. 1986. Experience and Its Expressions. In Turner and Bruner (Eds.) The Anthropology of Experience. Univ. of Illinois, Chicago.
[9] Federico Campagna. 2018. Technic and Magic. Bloomsbury, London.
[10] John Dewey. 2016. Having an Experience. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA.
[11] John Dewey. 2013/1925. Experience and Nature. Mccutchen Press.
[12] John Dewey. 2005/1934. Art as Experience. Perigee Books.
[13] John Dewey. 1938 Experience and Education. Kappa Delta Pi
[14] Paul Dourish. 2001. Where the Action is. MIT Press, Cambridge, MA.
[15] Anthony Dunne and Fiona Raby. 2013. Speculative Everything: Design, Fiction and Social Dreaming. MIT Press, Cambridge, MA.
[16] John Falk and Lynn Dierking. Learning through Digital Technologies. In Tallon and Walker (Eds.) Digital Technologies and the Museum Experience. Alta Mira Press.
[17] Kevin Farnham and Patrick Newbery. Experience Design: A Framework for Integrating Brand, Experience, and Value. John Wiley & Sons.
[18] John Fass. 2017. Representing Experiences of Digital Systems: The Design and Use of Externalising Models. Ph.D. Thesis, Royal College of Art.
[19] Jodi Forlizzi and Katja Battarbee. 2004. Understanding experience in interactive systems. Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, Cambridge, MA, USA, August 1–4, 2004
[20] Jodi Forlizzi and Shannon Ford. 2000. The Building Blocks of Experience: AnEarly Framework for Interaction Designers. Proceedings of DIS ’00, Brooklyn, New York.
[21] Michel Foucault and Duccio Trombadori. Experience Book. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA
[22] Friedman, K. (2003) Theory construction in design research. Design Studies 24(6).
[23] Uday Gajendar. 2008. Experiential Aesthetics: A Framework for Beautiful Experience. Interactions 15(5), 6–10.
[24] Bill Gaver, Anthony Dunne and Elena Pacenti. 1999. Cultural probes. Interactions Jan-Feb 1999, 21–19.
[25] Arjan Geven, Johann Schrammel and Manfred Tscheligi. 2006. Narrations and Storytelling as Methodological Key Elements for Studying User Experience. UX WS NordiCHI’06: COST294-MAUSE.
[26] J.J. Gibson. 1979. The Ecological Approach to Visual Perception (Ch. 8: The theory of affordances).
[27] Elizabeth Goodman. 2014. Design and ethics in the era of big data. Interactions XXI.3 (May-June 2014), 22–23.
[28] Colin M. Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L. Toombs. 2018. The Dark (Patterns) Side of UX Design. n Proceedings of CHI ’18 CHI Conference on Human Factors in Computing Systems Montreal, QC, Canada — April 21–26, 2018. ACM Press, New York, NY, 226–236. DOI:https://doi.org/10.1145/3173574.3173844
[29] Byung-Chul Han. 2017. The Scent of Time: A Philosophical Essay on the Art of Lingering. Polity Press.
[30] M. Hassenzahl. 2003. The thing and I: Understanding the relationship between user and product. In Funology. Springer, Netherlands, 31–42.
[31] Stefan Hellweger, Xiaofeng Wang, Pekka Abrahamsson. 2015. The Contemporary Understanding of User Experience in Practice. arXiv:1503.01732
[32] Kasper Hornbæk and Morten Hertzum. 2017. Technology Acceptance and User Experience: A Review of the Experiential Component in HCI. ACM Trans. Comput.-Hum. Interact. 24, 5, Article 33 (October 2017), 30 pages. https://doi.org/10.1145/3127358
[33] William James. 2016. Consciousness. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA
[34] Caroline A. Jones. 2016. Modeling. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA
[35] Caroline A. Jones, David Mather and Rebecca Uchill. 2016. Preface. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA
[36] Victor Kaptelinin. 2018. Technology and the Givens of Existence: Toward an Existential Inquiry Framework in HCI Research. In Proceedings of CHI ’18 CHI Conference on Human Factors in Computing Systems Montreal, QC, Canada — April 21–26, 2018. ACM Press, New York, NY, 226–236. DOI:https://doi.org/10.1145/3173574.3173844
[37] Victor Kaptelinin and Bonnie Nardi (2006) Acting with Technology: Activity Theory and Interaction Design. MIT Press, Cambridge, MA.
[38] Elvin Karana, Owain Pedgley and Valentina Rognoli. 2015. On Materials Experience. Design Issues: 31(3), 16–27.
[39] Joseph ‘Jofish’ Kaye. 2007. Evaluating experience-focused HCI. CHI EA ’07: CHI ’07 Extended Abstracts on Human Factors in Computing Systems.
[40] Joseph ‘Jofish’ Kaye, Elizabeth Buie, Jettie Hoonhout, Kristina Höök, Virpi Roto, Scott Jenson, Peter Wright. 2011. Designing for user experience: academia & industry. CHI EA ’11: CHI ’11 Extended Abstracts on Human Factors in Computing Systems
[41] Leah Kelly. Self of Sense. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA
[42] Hannu Korhonen, Juha Arrasvuori, Kaisa Väänänen-Vainio-Mattila. 2010. Let users tell the story: evaluating user experience with experience reports. CHI EA ’10: CHI ’10 Extended Abstracts on Human Factors in Computing Systems.
[43] John McCarthy and Peter Wright. 2004. Technology as Experience. MIT Press, Cambridge, MA.
[44] Helena M. Mentis, Jarmo Laaksolahti, Kristina Höök. 2014. My Self and You: Tension in Bodily Sharing of Experience. ACM Transactions on Computer- Human Interaction (TOCHI): Volume 21 Issue 4, August 2014. Article №20.
[45] B. Joseph Pine IIJames H. Gilmore. 1998. Welcome to the Experience Economy. Harvard Business Review, Jul-Aug 1998.
[46] Maurice Merleau-Ponty. 1945 Phenomenology of Perception. Routledge.
[47] Alva Noë. Understanding. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA
[48] Željko Obrenović. 2015. Design as a political activity: borrowing from classical political theories. Interactions 22(6), 42–45.
[49] Don Peterson and Mark Levene. 2003. Trail Records and Navigational Learning. London Review of Education 1(3), 207–216.
[50] Helga Schmid. 2019. Circadian space: Biotemporal and sociotemporal living space. In Harriss, H., Brooker, G. and Walker, K. (Eds.) Interior Futures. Crucible Press.
[51] Helga Schmid. 2017. Uchronia: Time at the Intersection of Design, Chronosociology and Chronobiology. Ph.D. Thesis, Royal College of Art.
[52] Tino Seghal and Arno Raffeiner. On Experience. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA
[53] Nathan Shedroff. 2009. Experience Design: A Manifesto for the Design of Experiences. New Riders.
[54] Lucy A. Suchman. 1987. Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge Univ. Press.
[55] Yasuyuki Sumi, Masaki Suwa and Koichi Hanaue. 2018. Effects of Viewing Multiple Viewpoint Videos on Metacognition of Collaborative Experiences. In Proceedings of CHI ’18 CHI Conference on Human Factors in Computing Systems Montreal, QC, Canada — April 21–26, 2018. ACM Press, New York, NY, 226–236. DOI:https://doi.org/10.1145/3173574.3173844
[56] Alex Taylor. 2015. After Interaction. Interactions 22(5), 48–58.
[57] Victor W. Turner. 1986. Dewey, Dilthey, and Drama: An Essay in 33 the Anthropology of Experience. In Turner and Bruner (Eds.) The Anthropology of Experience. Univ. of Illinois, Chicago.
[58] Rebecca Uchill. 2016. Mediating. In Jones, Mather, Uchill (Eds). Experience: Cognition, Culture and the Common Sense. MIT Press, Cambridge, MA.
[59] Harry van Vliet, H. and Ingrid Mulder. 2006. Experience and Design: Trojan Horse or Holy Grail. In Proceedings of User Experience, second COST294 international workshop held in conjunction with Nordichi,. 57–62.
[60] Kevin Walker. 2010. Designing for Meaning Making in Museums: Visitor- Constructed Trails Using Mobile Technologies. Ph.D. Dissertation, University of London.
[61] Kevin Walker, Suzy Hogg and David di Duca. 2013. Mechanics of wonder: scripting visitor experiences with technology. Proceedings of NODEM 2013, Dec 2013, Stockholm.
[62] Mikael Wiberg, Hiroshi Ishii, Paul Dourish, Anna Vallgårda, Tobie Kerridge, Petra Sundström, Daniela Rosner, and Mark Rolston. 2013. Materiality Matters — Experience Materials. Interactions: 20(2), 54–57.
[63] Wiener, N. 1950. The Human Use of Human Beings. Eyre & Spottiswoode, London.
[64] Terry Winograd and Fernando Flores, Understanding Computers and Cognition: A New Foundation for Design, (220 pp.) Norwood, NJ: Ablex, 1986. Paperback issued by Addison-Wesley, 1987.
[65] Niall Winters, Kevin Walker and George Roussos. 2005. Facilitating Learning in an Intelligent Environment. Proceedings of the IEE International Workshop on Intelligent Environments.
[66] Peter Wright and John McCarthy. 2010. Experience-Centered Design: Designers, Users, and Communities in Dialogue. Morgan & Claypool.
[67] Peter Wright, Jayne Wallace and John McCarthy. 2008. Aesthetics and experience-centered design. ACM Trans. Comput.-Hum. Interact. 15, 4, Article 18 (November 2008), 21 pages. DOI 10.1145/ 1460355.1460360 http://doi.acm.org/10.1145/1460355.1460360
[68] Mahnaz Yousefzadeh. 2013. Can Interaction Design Civilize the Experience Economy. Interactions 20(4), 44–17.
69] Caroline Yan Zheng. 2018. Affective Touch with Soft Robotic Actuators — A Design Toolkit for Personalised Affective Communication. In Proceedings of ACM CHI Conference on Human Factors in Computing Systems. 21 April 2018 Montreal.
