Universal Immersive Design | Accessibility VR Meetup Recap

Doug North Cook headshot

On April 16, 2020, Accessibility VR hosted a Meetup entitled “Universal Immersive Design.” Presenter Doug North Cook, founding faculty for the Immersive Media programs at Chatham University and lead instructor for the Immersive Design Residencies at Fallingwater, described a set of best practices that everyone should consider for accessibility in virtual reality.

Here are some of our highlights from the presentation:

Transcript: Universal Immersive Design - Doug North Cook

DOUG: Thanks, Thomas. Hi, everybody. So yeah, I'm Doug North Cook. I am a professor at Chatham University in Pittsburgh and run our immersive media programs there.

The core of that being an undergraduate program focused on equipping students to build the future of immersive applications. I've been working in immersive technology for about 7 or 8 years now. And before that, I was doing web development and software development, and now I'm focusing on accessibility work as well. So I am a designer.

But my hobby, passion, is architecture. I have spent the last several years working with the Fallingwater team, and I run the design residency programs at Frank Lloyd Wright's Fallingwater, outside Pittsburgh. And I would actually normally be up there today. It would be the start of day one of our spring immersive designer residency at Fallingwater.

And I'm very sad not to be there. But I would like to think that tonight can be a kind of nice placeholder for that event. So tonight what I'm going to be talking about is universal design. Which is a kind of design framework and philosophy that comes out of architectural design, and (inaudible) and this is my attempt to try to synthesize the core seven principles of universal design and apply them and talk through how they can be used when we're thinking about designing virtual environments.

A lot of my work is trying to synthesize architectural theory into immersive experience design, understanding that when we are designing virtual spaces, we encounter a lot of the same problems that we encounter designing physical spaces, and there's already a lot of really amazing work that's been done when thinking about designing physical spaces. A couple of things I want to talk about first. Kind of starting off in the early days of VR and a lot of people still exist in this place, around a lot of mythology and a lot of ways of thinking about VR that I think are wrong and miss the impact of the medium.

So I just want to talk about the VR mythos for a second. This is something that I hear a lot. And this to me is one of the biggest barriers to accessibility work in VR. This is a statement I hear a lot when I'm working with companies.

People say that VR is an active and physical medium. When we're trying to work on issues of accessibility. People want to get hung up on full room-scale, full body interaction, and they get really focused on the tactility that's possible in VR and the locomotion that's possible in VR. But my answer to that and a lot of what we'll talk about is that VR is multisensory.

We also should be thinking about immersive audio. We should be thinking about trying to engage all of the senses. One of the things that we talk about a lot up at our design residency at Fallingwater is trying to engage the seven senses. Which is something that comes out of an architectural theory from Johani Palasma, that says that there are the five traditional senses, but then there are also two additional senses, one being proprioception.

The sense of your body in space, and vestibular perception, the fluid in your ear and the sense of balance. It's when we start to engage with immersive technology as this truly multisensory technology -- that it's not just visual. It's not just physical. But it's -- it engages the whole of us.

And that unlocks a lot of really exciting opportunities, but also a lot of challenges, when we're thinking about accessibility. This is something that I hear when I talk with a lot of these companies. That difficulty equals fun. This tends to be a way that people kind of scoff at accessibility design and frameworks.

Thinking that by adding more accessibility options, you are taking away from the challenge and the inherent fun of an experience. But I would say that arbitrary difficulty is just exclusivity and gatekeeping. And that really good experiences, especially really good virtual reality experiences, tend to have a really nice gradient of difficulty.

MIRABAI: I'm having to manually click on the speaker every few sentences to make the garbling go away.

DOUG: I'll talk about a few experiences that I think exemplify these in a really clear way. And from my perspective, universal design and accessible design is not just something that we do for people who have a specific ability that is restricted. It is something for ourselves and something for all of us. Universal design acknowledges that at some point, all of our abilities will change.

That we will all age. That we will all at some point get a temporary disability. We'll get in an accident. Our hearing may change.

We all change. So universal design is a framework that implores us not just to think about people who currently need accessibility features in order to interact with technology. But also thinking about our future selves. So the first of these seven principles in universal design is equity in design.

And this is, to me, one of the best cases for thinking about diversity of team structures, when we're approaching difficult design problems. That if your design team does not have voices that represent a variety of abilities, a variety of backgrounds, that you're going to have a lot of challenges when you're trying to design meaningful and accessible experiences. And the only way that you can get around that, but that doesn't quite replace it, is to do really, really robust user testing, with a wide variety of users.

And there are a lot of groups that do really great work in this area. Able Gamers, specifically, has a really amazing user testing group for digital experiences. And there are a lot of other groups as well. But if, like a lot of immersive studios, if the makeup of your studio is only representative of a couple of different backgrounds, and you have no one with a varied ability on your team, you really have to make sure you're doing your due diligence in bringing in serious and robust user testing, because otherwise you're not just missing the opportunity to create a more accessible experience for a broader audience.

You're also missing the opportunity to possibly create a more accessible experience for your future self.

MIRABAI: Going into another tab helped with the distortion.

DOUG: The next is looking at flexibility. This is something that, again, I would say all seven of these benefit really well from having a really diverse group of people working on your project. So when we're talking about flexibility, we're talking about the need for your virtual reality experience to not be myopically focused. And have the ability for the user to interact with it in a variety of ways.

So flexible user interactions in VR might be -- when we're thinking about reaching for an object that's out of reach, do I have to go and pick that object up? Do I have to physically go and get it? Or am I able to reach my hand out and have it come to me? Am I able to have a gaze-based interaction?

How many different ways can I perform each action? And these flexible options don't just allow for greater accessibility. They also allow for the possibility of more emergent game play for users to develop and create their own new interaction combinations inside of an experience. A lot of experiences miss this opportunity and they go for a single mechanic, a single way of interacting with an object.

But when we do think beyond that, to variety, to greater accessibility and types of interactions, we're allowing not just a wider variety of players, but we're also creating a richer experience. So simple and intuitive. And this -- if you've played any of the top selling virtual reality games, or you've seen some of the best in class virtual reality training experiences, you know that this is exactly how they operate. Interactions are kept to a bare minimum.

Most controller usage is kept to a single button press. And most of what is in the world is immediately discoverable. I'm sure that a lot of you are familiar with Don Norman, the designer and writer. And he talks a lot about discoverability, affordances, and signifiers.

And how an object, how an interaction in an environment, is able to be discovered by a user. And really, really good experiences don't need a lot of labels. They don't need a lot of user interface. Really good experiences rely on intuitive interactions that the user just instinctively understands how to perform that action.

But I would say: This is a caution. And Norman actually recently wrote an article kind of looking at himself now, because he now is getting into his 70s, and he's realized that a lot of the world that he helped to design is no longer accessible for him. And there is a misnomer with simple and intuitive design that, because something is simple, and almost invisible, that it can become actually invisible to users.

And I see this a lot with people that I know, who have been using Android phones their entire life, and then they get an iPhone. And because they haven't built in all of those intuitions for how to use that operating system, the operating system for everyone who's been using it feels simple and intuitive. But for someone who has never used it before, it's not. And all of the signifiers are buried behind these trained intuitions.

And those same kinds of things can happen in VR. As we continue to develop design languages and understood best ways of doing things, those become ingrained in the culture, and they become understood by people who use VR on a regular basis, but they're not going to be understood by the people who are getting into VR for the first time. And this is a really big issue. Because I've worked with a lot of companies who are building enterprise-grade training experiences.

They're bringing in some best practices from VR games and game design. But then the users that are interacting with those experiences -- they're not gamers. They've never used VR before. It might be a crane operator in his 50s, who is jumping into a simulator, jumping into a training experience.

And they're not gonna have those same kind of pregained intuitions and backgrounds that we can expect from a regular VR user. Perceptible information. When all of us working in accessibility talk about this a lot -- that we need to make sure that critical information, that keeps a user moving through an experience, can be gained. And that that can be gained from a variety of ways.

Right? So when we're talking about VR being multisensory, we need to rely on multiple senses to impart information. So we need to make sure that users have really good feedback for the completion of a task, for any sort of dialogue that we have subtitling. That we have good audio feedback.

Good haptic feedback. Good visual feedback. And that all of that is formatted in a way that's accessible to a variety of users. Right?

That you're taking into account colorblindness. That you're taking into account people with hearing impairment. And this gets really complicated, right? Because VR -- you think you can rely on a certain number of things.

But you've got people who are wearing glasses. You've got people who have restricted field of view. And not only that. When we're working in VR, we're also dealing with people on a wide range of headsets, with a wide range of resolutions.

So text that is really crisp and clear on a Valve Index might not be as crisp and clear on a first generation Rift. So you really have to think broadly, and again, do make sure that you're doing really good testing. Not just with a variety of users. But then also with a variety of hardware.

And this is why making really good, really sticky loved virtual reality experiences is so difficult. Okay. Tolerance for error. I like to tell a couple of stories about this.

The first is: I took -- a friend of mine. Put him into VR for the first time. And any time I put someone into VR for the first time, I always ask them what kind of experience they want to try. He was really excited to try something really, really intense.

So I put him into Raw Data, which is a robot shooter. And he got so into the experience that he was actually, like, hiding on the floor from these robots, behind a virtual box. Was bracing himself on the floor. And then a robot had snuck up behind him.

And when he turned around to look at the robot, he got so afraid that he just stood up and took off running in the room, straight into a brick wall. Headset first. And... He was fine.

The headset was fine. But this happens all the time. Users make errors in judgment. And those errors are not just about the software.

They're also about human behavior. That we expect people to interact in a certain way in the virtual experience, but you can't anticipate all possible interactions. And you also cannot anticipate the size and scale of someone's room or physical environment, and you also cannot anticipate their physical reaction. So if you create an experience that has a high possibility of motion sickness, if you have an experience that has a high possibility of someone being able to end up in an environment where they lose their vision or they lose the correct balance in their hearing, you could actually cause a physical disturbance for that person.

So we need to try to anticipate those. Those errors. We need to put parameters in place for them. If the user tries to stick their head inside of a character in an experience, or tries to put their face through a desk, what's gonna happen?

Right? Are you gonna show them the glitched geometry of the inside of a 3D model? Which might cause them to have some sort of vestibular mismatch, immediately? Because they're no longer seeing proper geometry?

They're seeing a bunch of glitch shapes in different planes? Or are you going to have a system in place to direct them to pull their view back to the proper distance? So it's both trying to anticipate and understand what errors are possible, both within your experience and with your users. And then designing for those errors.

Physical effort. So this is an interesting one. When we're talking about VR. And going back to this idea that VR is an inherently physical, very active medium.

And while that's true, that is not always what users can do. And it's not always what users want to do. I actually -- in a workout, a few days ago, sprained my rotator cuff. So I actually don't have full range of motion in my right arm right now.

So there are a lot of virtual reality experiences that I currently would have a really hard time playing through to completion. Because there are objects that are too far out of reach. On a high shelf. In a room.

So you really need to think again... When we're solving for tolerance of error, when we're solving for intuition, that we need to be thinking about the physical effort that is required to perform this action. Especially when we're thinking about experiences that might have a lot of repetitive motion. Where you have the possibility of actually causing harm or injury.

Or if we're really trying to think about making an experience more accessible to a wider audience. Because again, not all users can perform these actions. But also, a lot of times users just don't want to. A lot of times, somebody will want to play your VR experience in a more relaxed posture.

Or sitting in a chair. So again... When we're thinking about designing for someone in a wheelchair, we're also able to design for users who just want to play an experience seated. So in accommodating an accessibility request, oftentimes we're also accommodating a wider user request as well.

Size and space. So when I was talking a little bit about tolerance for error, and talked a little bit about physical space, this is a really important consideration. And this is not just about creating better access for people with a variety of abilities. This is about creating a better experience for people who live in a variety of spaces.

Somebody that was in here before said that they were in Brooklyn. I have a lot of friends who live in Brooklyn, in New York, and have very, very small apartments, and they can't have a full room-scale set up with the maximum tracking volume for a Vive or a Valve Index or another large scale VR setup. But a lot of really great experiences are built to take use exclusively of these spaces. But a lot of really great experiences will accommodate a variety of room sizes, room scales, and also player sizes.

I remember putting my niece, who was six at the time, into a very short experience. And just trying to show her something in the goggles, very quickly. But she couldn't see it. Because she was too low on the ground.

And the headset was clipping her through the floor. So again, making sure we're accommodating not just a variety of physical spaces, but a variety of sizes of users. Okay. So those are the 7 principles of universal design.

So I'm just gonna run back through them really quickly. So equity in design. Flexibility. Simple and intuitive.

Perceptible information. Tolerance for error. Physical effort. Size and space.

And these seven principles are employed in architecture constantly. This is how people approach designing buildings all over the world right now. And this is how I and my team have started approaching our design for virtual experiences. At the university and with the companies that I work with.

That we use those seven principles as a filter to ask ourselves hard questions during our design process. So I just want to give a couple of really quick examples from VR experiences that a lot of you I'm sure are very familiar with. Just to show what some of these things look like. So Beat Saber, which is currently now the top selling virtual reality experience that exists, they've done incredibly well, and a lot of people like to theorize about why they've been as successful as they have.

And I think there are a lot of reasons for that. But I know that a large reason is because they have some of the best accessibility and user features of any virtual reality experience. They have colorblind mode. They have one-handed mode.

They have no fail mode. They have dozens and dozens of user options and reconfigurations to allow people to play from a variety of postures, in a variety of room sizes, with a variety of equipment. And this has allowed them to become the single most ubiquitous virtual reality experience. Because it's something that almost any user can play in some form.

That is not true of almost any other virtual reality experience. And a lot of people would argue that Beat Saber is a full-body, full-room-scale rhythm game. Which it can be. But it can also be played in a very different way, in a very -- with very different equipment.

By a wide variety of users. One of my favorite things that they did, if you haven't seen -- there's a short video that they posted about a system that they built, with the company SubPack, to create a haptic version of Beat Saber, for the deaf and hard of hearing community. And then they took that and demoed it at a couple of conferences, where players could play Beat Saber without any audio, just with a haptic system.

And that to me really embraces this idea that even this rhythm game could be a multisensory experience. And be played just with haptics. So the other is Vacation Simulator. From Alchemy Labs.

They have a really great blog post on the Alchemy Labs blog, talking about after they released Vacation Simulator, and adding in accessibility features that they either didn't have time to get in, or that they missed. And one of those that I really love is that they went through and they made sure that people could complete the game with a single controller, seated. And that you could play through all of Vacation Simulator with one controller, seated in a chair.

Which was really great and really opened up this experience, which was a really calm, relaxing, and fun experience to a huge range of users, but also to people who just want to sit down and maybe they're out of batteries in one of their controllers. So again, thinking about broadening and opening this up to a broader group. And the last is Virtual Virtual Reality. From TenderClaws.

TenderClaws are one of my favorite virtual reality studios. If you haven't played their newest experience, The Under Presents, I think it's the best thing that's come out for VR in the last year. But one of their earlier experiences, Virtual Virtual Reality, which originally came out on the Google Daydream, and is now out on Rift and Quest, they were one of the first groups that I saw that did spatialized subtitling. Where they actually took the subtitles for all of the dialogue in the game, and put them into three-dimensional space, and made them a part of the environment.

So not just adding in subtitles that accommodated, but adding in subtitles that actually become a built part of the environment, and really enhance that experience. Right? So it's not just about making the features exist. But it's about finding ways to have those features enrich the experience.

And be a part of the experience. So there is so much more left to do. And I could talk about examples and ideas and things all night. But this, I think, is some of the most exciting work that needs to be done in virtual reality, still, is to continue to figure out how we can make this medium as open, accessible, and exciting for as many kinds of people as possible.

And that's all I've got. Thank you very much.

THOMAS: Thank you. That was great, Doug. Awesome. So I think now we'll see if anyone here in the audience would like to ask a question.

If you want, you can just unmute your mic. Ask the question. And we'll take about... 10 to 15 minutes for questions, if folks have questions.

Hi. Dylan Potts over here in the back right. Can you hear me okay?

DOUG: Yeah.

Hi, Doug. I'm curious. From your perspective, what are the barriers -- outside of just motivation, for people -- developers that are motivated to have accessible products, what are the barriers that are stopping them from doing so? What are the resources that they would need that just don't exist right now?

DOUG: Yeah, sure. Yeah. So Dylan's question was: What are the barriers that exist for developers and studios that want to add more accessible features to their experiences? I mean, I would say...

The two biggest barriers for most people are money and time. Because most people don't prioritize accessibility features and accessibility tools. Both from a budget perspective and also from a project management perspective. So it's usually those types of features that get sidelined in development.

But again, I think when you look at the experiences that have been really, really successful, a lot of those have really robust accessibility features right out of the gate. And from my perspective, that's what can allow an experience to become incredibly sticky and really capture audience. Right? Because right now, when VR is still growing, and expanding, I think about a family holiday moment for a VR experience.

I'm thinking about a VR experience that I'm going to show to my grandmother and to my nephew, when I'm home for a holiday. Right? What are those kind of core experiences that I can have anyone in my family play? Anyone -- any of my colleagues play?

And so yeah. I think a lot of people sacrifice them for time and for some savings on the budget side. But I also think that a lot of people are restricted by just a misunderstanding. Of what this type of design can do.

Which is why I try to take this approach to it. That it's not just about making experiences accessible for people with a range of abilities. That it's really about broadening your audience, and that that isn't just about accessibility. It's about something bigger than that.

Yeah.

THOMAS: Great. Any other questions? I'll ask one. So the Vacation Simulator rework that you mentioned from Owlchemy -- did they say anything about the level of effort to do it?

Was that anything they commented on?

DOUG: Yeah. They don't talk about that in their blog post, but from talking to some of my friends there, it was a pretty big lift. After release, they really dedicated the majority of the team to working on their next big update, which was almost exclusively accessibility features. And one of which they added in -- a really incredible and adaptable subtitling tool.

That does full overlay, and it's fully spatialized. I think they took what TenderClaws had done in Virtual Virtual Reality, and built it out into a robust framework for doing VR subtitling. So if you haven't tried Vacation Simulator, I think it's worth trying, just to see how they do subtitling, because it's really, really elegantly done. But one of the biggest payoffs for them and for other studios is that if you have a really robust and really elegant subtitling solution, it makes your localization costs come way down.

Because you can keep your English language -- or wherever your experience developed -- you can keep native language audio, but then you can do localization just in text. If you have really robust subtitling.

THOMAS: That's cool. Anyone else have a question?

DOUG: We can also probably take questions in the chat, if people want to type anything too.

THOMAS: Sure.

Hi. Can you hear me?

THOMAS: I think I hear someone speaking, but it sounds quiet for me.

DOUG: I don't hear anyone. If you do have a question, and you're trying to talk, you can come up to the front. And then we can try to hear you.

All right.

DOUG: Oh, here's a question. What are some of the best games for wheelchair users? That is a really good question. Yeah.

I mean, I think... Again, the Owlchemy Labs games. Vacation Simulator, specifically, has enough robust movement and object interaction tools to be able to be played by a variety of users. Actually, one of my favorite VR experiences in the last year is Tetris Effect.

Which is just played with a standard controller. Fully seated. I think it's one of the more elegantly done kind of 2D games in 3D. That's a really great one.

But I also think that Beat Saber, played in -- with some of their accessibility features turned on -- is fully navigable as well. Again, I think some of these titles have done a really good job of allowing users to customize enough options that, yeah, you might have to spend a little bit of time doing some calibration. But once you do, you can do a full playthrough on some of these experiences. And the feedback that I've gotten from users is that those have been really positive experiences.

Yeah. Great question. I saw a question here. Let's see.

THOMAS: So one of the questions or comments was around gaming discussions. And I think the sound effects usage. By a blind gamer. A comment on whether that could be a good option.

But maybe the question from Rahmat is around the audio design experiences? Something like that? Ambience for outdoor and indoor being different, the enemy appearance in characters, richer sound design questions?

DOUG: It's interesting. I haven't seen very much done with fully spatialized audio virtual reality experiences. But it's something that... We had actually started exploring with a couple of projects we were working on this past year, and it's something that I'm really interested in.

I think a lot of experiences don't lean as far into the spatialized audio possibilities, because again, audio is something that gets -- oftentimes gets sidelined in budgetary constraints. And doing really sophisticated spatialized audio can get really expensive. And really difficult. But yeah.

I think when we start to strip away the visual component of a game, and you think about a virtual reality game that relies on haptics and audio, I think there's something really exciting to explore there. That, again, isn't just an exciting experience for blind users. But is an exciting experience for a variety of users. Right?

So yeah. No, it's great. That's a great comment. VR and Screen reader compatibility.

Interesting. Yeah. So the question in the chat is talking about VR and screen reader compatibility. I mean, I would love for there to be kind of universal platform integration of some of those kinds of tools.

Unfortunately, unless it's baked into the experience or into the platform, it's gonna be really hard to get that compatibility. And I think it really takes users asking for those kinds of features. But also takes someone being on a team at one of those companies, who is advocating for that happening. But I do think...

Yeah. Really, those kinds of tools and features -- like, the best accessibility tools and features will be baked in at the platform level, whether that's on Oculus or on Steam or on Playstation VR. Because putting the burden on individual developers to build robust accessibility tools is really difficult. Because again, it's expensive.

It's time-consuming. And I think a lot of people really would love to see more support from the platforms. Which is something that I'm constantly trying to advocate for.

THOMAS: Well, great. So thanks, Doug. I think we're gonna just transition here to getting into the... Oops.

Moving forward here. To the just chat and discussions. And so people, if you can come up to Doug after the talk today, and if you have questions that you would rather ask in person, he's gonna be able to stay around longer with us. But again, let's all say thanks to Doug.

DOUG: I'll come down from the sky.

THOMAS: Thank you, Doug. And before we close out here today, just for the presentation part, I want to let you all know that we're planning to have just a purely social meetup. Same time next week. And Mike Gifford, who I believe is in the room here, he'll be co-hosting that.

So that's just a new idea we want to try out as well. To explore. And we also have our next presentation scheduled for May. That I'll announce on the Meetup page at the conclusion.

So one more time, let's all say thank you to Mirabai Knight from White Coat Captioning for the awesome captions that have been provided today. And to Joly MacFie from the Internet Society Accessibility Special Interest Group for the great work of getting our stream on YouTube. And so all of the presentations we do at a11yVR will also be available on YouTube afterwards, if you missed one, or if you want to go back and review what was presented.

So again, thank you, everyone. And now I'm gonna be posting links into the chat. We did create some breakout rooms that are also linked on the Meetup page. We had feedback last time that for some people, when lots of people were talking in the room, it was distracting.

So we do have three other rooms that we can join or cycle through. If we want to have smaller discussions. And so we're just experimenting with that today. I'm gonna paste those into the chat.

And let's enjoy connecting. And thank you for coming today.

DOUG: Yeah. Thanks, Thomas!