March 15, 2019 by Ryan Howard

A Conversational Framework for Integrated Experiences

Sometimes, the job of facilitating experiences can be a real experience itself. Especially when a group of stakeholders is, ahem, discussing what exactly it is that we need to deliver. Design firms and integrators will often begin with a needs analysis or discovery phase in which they look, listen, and then turn around a report and recommendations. It’s a proven approach that keeps projects moving. But if a project doesn’t exist yet — and this happens quite a bit within large organizations — initiatives spin up, stakeholder groups are assembled, and conversations run the gamut of well-informed to highly speculative. If the initiative is straightforward, like finding a new copy machine contract, the conversation will tend to sort itself out, being that copiers are a well-established capability. But if the initiative is more like “we need to improve our _____ experience,” conversations can get murky.

It can be challenging to talk about experience because the word itself is so pervasive in daily life. Unlike a term that has a specific and unchanging meaning, such as, say, “local area network,” we talk about experience contextually, with varying specificity, and with individual biases. We often combine other words to narrow the scope of the term to a discipline: user experience, customer experience, workplace experience, or, for the AV world, integrated experience. However, this doesn’t give us tools to talk about the experience qualitatively, and it’s common to see multiple disciplines within a single project.

The Framework

Intent: Emotive/Informational

We facilitate experience through the application of human-scale human–computer interaction (HCI). At the root of that value proposition is the conveyance of information. How we go about conveying information is largely how we impact experience, so this first characteristic, intent, is a natural starting point. It’s also interesting how deeply actionable it is, even in a fairly abstract stage.

Do we need to inform or create emotion? Recognize intent is deeply actionable because it’s deeply nuanced. To inform requires visual and auditory acuity, which translates to resolution and system performance. Informational systems are conveying language: ideas that are already codified. Purely information systems look like airport flight status boards and PA systems. They are emotively neutral.

Emotional systems are something different entirely, where we are feeding human perception into our intended outcome and that depend heavily on psychological, social, and cultural factors. Purely emotive systems are rare in a commercial environment, as they best serve artistic functions. But there are some great examples of architectural media that are heavily weighted toward the emotive end of the spectrum. In these installations, you may never see or hear a word, but imagery, light, motion, and soundscapes are combined to create an atmosphere and to establish an identity of place. While standards and best practices are definitely involved in the successful execution of these projects, there is enormous variety in how HCI systems are used materially.

Contextual Priority: Tangential/Immersive

Contextual priority describes how canvases are placed in space and relative to the audience’s sensory focus. In a tangential application, as in the case of electronic wayfinding signage, we are supplementing the audience’s awareness of the environment. With signage, we need to anticipate where the audience might look and might need to go, but we don’t typically place a physical impediment in their path. It’s still important information, but it shouldn’t cause someone to trip over the person in front of them.

In an immersive application, we are trying to shape the total experience by controlling the total environment. Theaters and auditoriums commonly do this. There is a focus to the space, and if you look away, cues such as the audience’s gaze nudge you back into the moment. I believe this is an underexploited experiential characteristic within integrated experiences, though we see its potential emerging within workplace theory, where the idea of activity zones is driving architectural and real estate design. We have the ability to build systems that can be tangential and immersive at different times. We can provide layered experiences through the choreography of HCI touchpoints and behaviors.

Engagement Time: Transient/Captive

Engagement time is closely related to contextual priority, in that there is a natural association of transient to tangential and captive to immersive. Those associations can reverse, though. Think about standing in line for an attraction at a theme park. That’s a captive audience where tangential media (digital signage or projection mapping) can be used to tell a story. Transient opportunities can be immersive as well. The Fremont Street Experience in Las Vegas is an example.

Contextual priority is often used to address a constraint around engagement time. Experiential retail demonstrates this as it tries to grab shoppers’ attention and drive brand messaging and engagement quickly and efficiently.

Spatial Scale: Personal/Architectural

I think of spatial scale as an aggregate of the scale of all the parts. Systems can have more than one canvas and can be composed of elements of different scales. For this reason, it’s important to break the range down so that the different break points may be understood. I describe the full range as wearable to handheld to desktop to human to furniture to architectural. That may seem like a lot (I generally refer to wearable, handheld, and desktop as simply personal), but each of those scales tends to establish a different user experience capability, progressing from the intimate through the communal.

Today, even in a modestly sized built environment, we find mixed canvas scales. One of the best opportunities we have is to better utilize those canvases cohesively and with minimal friction.


Spatial Organization

Spatial organization recognizes that a mix of canvases and canvas scales act as a single system in facilitation of a common experiential outcome. A large, contiguous videowall creates a different impact than scattered monitors such as those in a sports bar.

As soon as multiple canvases are introduced into a design, spatial organization becomes a quality that must be managed to assure a cohesive experience. Consistent sound levels, properly localized audio, consistent color rendering, and complementary pixel pitches are among the elements that should be considered. But spatial organization can be a powerful influencer of both capital and operating expense when scale requirements are expansive. Total pixel count in a system is one of the most significant cost drivers. By utilizing smaller canvases so that fewer pixels are distributed across a given area, it’s possible to create cost-effective systems of large spatial scale. Returning to the sports bar example, a dozen commercial monitors are likely to be a less expensive solution than a single large LED screen. Creative canvas organization and orientation also provides a way of disrupting the ubiquity of the 16:9 aspect ratio that dominates the flat-panel-display marketplace. Architectural media and digital signage often harness this principle by utilizing smaller displays in clusters or ribbons, or mixing display sizes.

Interactivity: Static/Interactive

There is so much screen interaction in our daily lives, it’s easy to assume that any interactive experience (IX) system should automatically be interactive. A truly interactive system can be a powerful tool for collaboration or presenting explorative narratives. But the alternatives to interactive — static as the extreme opposite and a responsive middle ground — may be more appropriate. A mobile device – personal scale – is constrained by canvas size, so it’s a baseline necessity to provide interaction that allows the user to navigate different features. This establishes the need for an entire discipline, user experience (UX), to tune those interactions for an array of intents and desires. IX begins to share in the UX role when human-scale canvases are introduced, where we have different concerns than at a personal scale. In many cases, the best user interaction may be none at all.

Programming: Human Action/Machine Intelligence

Programming directly asks the question, “How are we supposed to feed this beast?” IX systems are voracious. Digital signage and architectural media providers have become adept at developing powerful content management systems that play out media based on templates and rules. Generative systems render complex animations or images in real time or “just in time,” feeding data streams into creative algorithms. Sometimes, humans manage the entire process, especially where a constant awareness of audience sentiment is required. Think about how a club DJ might manage a set depending on how the dancefloor is reacting to music.

This extends into the collaboration space. Scheduling systems help to eliminate the tedium and cost of people managing hundreds or thousands of conference rooms in a global enterprise, and yet sometimes it’s helpful to have people in that scheduling mix, perhaps to ensure the best use of auditoriums and other high-performance presentation environments.

Business Dependencies: None/Intrinsic

Are there thousands of hours of video that we need to mine? Does the content need to track marketing campaigns? Are there strong vendor biases or client relationships that we need to factor into the design?

Business dependencies are a reality and they can significantly shape design approach. A requirement to track closely to marketing campaigns might drive an approach to workflow. A deep collection of video content might prioritize aspect ratio. It might be necessary to utilize a particular technology or vendor. Constraints drive good design, and establishing an early understanding of key dependencies will inform those constraints and shape the conversation in positive ways.


Impact: Forgettable/Memorable

Do the experiences we create need to be memorable? If brand awareness is the goal of the effort, yes. But should a command and control environment be memorable? Should a collaboration environment be memorable? This is a question around both the systems and the intended experience itself. In a world powered by hyperbole, it’s sometimes hard to navigate by anything but the brightest, the loudest, the most sensuous, or the most enveloping. In IX and in life we chase those peaks, challenging ourselves and each other. Our resumes lead with our biggest and best accomplishments, the grandest deployments, or the multimillion-dollar successes. Not every experience needs to be memorable, but every experience we are tasked with facilitating is valuable in some way. As a community we enable endless fleeting moments that power the commonplace. It’s easy to forget how hard it may be to achieve the forgettable. It often feels simple because we create a formula to repeatable success through best practices and corporate standards. Yet, the commonplace is in high demand because the commonplace has enormous impact on businesses. People have important conversations; they find their way through unfamiliar places; they keep informed while on the move — all through the seemingly banal outcomes of our efforts. Sometimes the most forgettable experiences are the most consequential.

Keep on Talking

Modern life is defined by the constant technological mediation of experience. IX plays a role in this mediation, delivering systems that are a mix of interfaces and behaviors. The ubiquity of mediation means that “integration” extends well beyond the boundaries of a single installation. Increasingly, we carry narratives through disparate experiences and moments. A fleeting interaction with a mobile device while in line for coffee may relate to an extended conference session with face-to-face interaction. Events may simultaneously span multiple physical sites and audiences. The success of distributed experiences is dependent on the ability to define them and to provide tools to facilitate them efficiently and resiliently.

The characteristics above form a framework for talking about experiences — specifically, integrated experiences. It’s not a checklist, formula, or standard. The value in early stakeholder conversations should be apparent, especially where the stakeholder group is entering unfamiliar territory with new complexities, new applications, or specialty deployments. There is also value in applying this framework to the commonplace, even retroactively, as it provides a critical lens that is humanistic rather than technical. In focusing on the human, we leverage tools beyond the technological. This framework is a part of an emerging discipline, but it is only one piece. Our role in the mediation of experience unveils questions around ethical practices, and our continuing requirement for physical interfaces between people and digital begs similar questions around ecological practices. As we elevate our value proposition, we must also elevate a critical discourse to drive the quality and integrity of our efforts. There is limitless opportunity ahead of us, and a never-ending conversation to be had. It’s sure to be quite the experience.

Ryan Howard is a technologist, designer, and artist who powers story-telling through technical systems. He is the founder of Storied Systems, an audiovisual and media technology consultancy that designs systems, strategies, and environments for storytelling at human scale. Ryan's wandering curiosity propels his interdisciplinary work, spanning experience design, product design, audiovisual systems design, software development, game development, broadcast media systems, architectural media, and his visual art practice.

About Ryan Howard

Ryan Howard is a technologist, designer, and artist who powers story-telling through technical systems. He is the founder of Storied Systems, an audiovisual and media technology consultancy that designs systems, strategies, and environments for storytelling at human scale. Ryan's wandering curiosity propels his interdisciplinary work, spanning experience design, product design, audiovisual systems design, software development, game development, broadcast media systems, architectural media, and his visual art practice.