How Can a Blind Person Use Virtual Reality?

Virtual reality is a very visual medium. It might surprise you to learn there are people who are blind or have low vision who enjoy virtual reality. Jesse Anderson is one of them. Unfortunately, many virtual reality experiences are not accessible for them and other disabilities.

Jesse shares advice for creating accessible virtual reality experiences for the blind and low vision. He also discusses efforts to improve accessibility in virtual reality.

How did you get your start in accessibility?

I have been legally blind my whole life and started using assistive technology in elementary school. Ever since I tried my first talking Apple II computer, I have been fascinated with technology and assistive technology. Throughout high school, I learned both DOS and Windows and used blindness and low vision programs like JAWS, ZoomText, Magic, Kurzweil 1000, and Open Book.

Once in college, I started getting more involved with accessibility testing, as well as training other students to use their own assistive technology. I worked in the university’s Disability Services office to help other blind and visually impaired students learn to use their screen magnification and screen reader software with course learning management system websites and other tools.

I would also work with the Instructional Technology Center (ITC) and other college professors to test course websites, documents, and other tools for screen reader accessibility, as there were several blind and low vision students attending the university at the time.

Ever since I was a kid playing Super Mario Bros., I knew I wanted to get a job in the gaming industry someday. I first thought I wanted to be a programmer, so I could make games. After taking a few programming classes in college though, I tell people, “The only thing I really learned in programming class is that I sucked at programming …

While testing programs and websites for accessibility in college, I figured out that I could look at technology and games from a more user interface angle and help developers make their apps more accessible. My technical writing and instructional design background also helped with this.

In 2015, I attended a session from Ian Hamilton on game accessibility at the CSUN Assistive Technology Conference in California. After hearing his talk, and speaking with him after the session, I was inspired to focus more on game accessibility. Ian gave me some great information and ideas and introduced me to a few people in the gaming industry.

Since then, I have worked with indie and AAA developers on making their games more blind and low vision accessible. I have also spoken at the Inclusive Design 24 (ID24) virtual conference, Game Accessibility Conference, and my local Independent Game Developers Association on game accessibility.

What’s your backstory?

I have been legally blind all my life and have been into gaming and technology for as long as I can remember. I remember playing the Atari 2600 for the first time and simply being fascinated that I could control something on the screen. Around the same time, I saw my first Apple II computer, with text-to-speech no less, and was equally fascinated. Shortly after, I got to try the NES for the first time, and I have been hooked ever since.

I have been using assistive technology since the Apple II days as well. I’ve also used a variety of screen magnifiers and screen reader programs since. I received my bachelor’s degree in information technology management with a minor in scientific and technical communication. Then, my master’s in instructional design. I have been interested in the accessibility area for many years, and feel that — while technology won’t solve every problem — it can be a huge equalizer in leveling the playing field for people with disabilities.

One trend I’ve loved seeing over the past several years is the inclusion of accessibility into mainstream devices and apps. What used to often cost several hundred or thousands of dollars is now available for free or for a small cost. The iPhone and its ecosystem of apps have opened up possibilities for many people.

All major computing platforms including Windows, Mac, iOS, and Android all include a variety of accessibility options. Home appliances, TVs, set-top boxes, and even game consoles are building-in accessibility. I’m excited to see this trend continue further and will continue to advocate for this type of accessibility.

Your YouTube channel is “Illegally Sighted.” What’s the story behind that and what do you cover?

That’s actually kind of an interesting story. Whenever I would meet someone, they could usually tell that I was visually impaired. They would always ask me right away about my vision. I got tired of giving the same rather dry explanation, so I tried to think of a more interesting explanation. Even though I know what the term means, I always thought the term legal blindness was rather odd. One day I thought, if I was legally blind, then I must be Illegally Sighted.

So, when people would ask me about my vision, I started my explanation by saying that I was illegally sighted, which often caught people off guard. I never got tired of the look on their face, as they were trying to parse what I just said, so I kept rolling with it. When trying to come up with a name for my YouTube channel, I remembered this little term and felt it would be perfect.

As for starting the YouTube channel, that also started as kind of a happy accident. Several years ago, a friend and I got really hooked on Minecraft for a while and we’re always sharing our files to show off our newest creations. This quickly became impractical due to the large file sizes. We also thought it would be cool to show off our creations to others.

One day, I decided to record a couple of short videos where I gave a tour of the medieval castle village I was working on. I then posted a couple more videos showing off a couple of games I was playing. I found that I rather enjoyed reviewing games like this, and thought it would be an interesting possibility for working my way into the games industry if even a little.

At the time, gaming videos on YouTube were starting to become very popular, and there were thousands of channels covering games in some way. If I was going to continue the channel, I would need to find a hook that differentiated my channel from the thousands of others. I decided to cover games and technology from a low vision user’s perspective. Things just kind of took off from there. I started the channel in May 2012, and it’s hard to believe the channel is almost ten years old now.

What are your favorite topics to cover on the channel?

My favorite types of videos are for mainstream games that include accessibility options and content to make them blind and / or low vision playable. While I enjoy audio games, and screen reader-accessible games designed specifically for the blind, many of these titles are more simplistic and don’t hold my attention for as long.

Don’t get me wrong, there have been some fantastic audio games, but I love the recent trend of more mainstream titles becoming playable to blind users. Some titles like Skullgirls (a 2D-fighting game) feature full native screen reader support on Windows. Other titles like Mortal Kombat 11, Minecraft Dungeons, Forza Horizon 5, and many recent Ubisoft games all have some level of blind accessibility options like narrated menus and other audio cues.

Sequence Storm and Eagle Island are a couple of other excellent examples of indie titles — both developed by one person — that have many great accessibility options. I love that I’ve been able to cover games from the smallest of indie titles to the biggest of AAA games, and more and more are starting to include at least some level of accessibility.

My favorite mainstream game though in recent years is definitely The Last of Us Part II (TLOU2). This PlayStation exclusive game is definitely not for kids, but its accessibility options are well beyond anything I’ve experienced in a mainstream AAA title. TLOU2 includes over sixty accessibility options and is playable from start to finish, on any difficulty level, by blind players.

Some options I especially found helpful were the menu narration, high contrast mode toggle, and the built-in magnifier. The coolest ones though were its navigation system for blind players. As a blind gamer, it is easy to navigate an open-world environment and locate objectives, enemies, weapons, and other items of interest. Now that Naughty Dog, the game’s developer, has set the bar so high, I can’t wait to see other mainstream games improve on this level of accessibility further.

I also have to mention a few game mods, as several of these have been released in the past couple of years. These game mods make existing games, otherwise largely unplayable, blind-accessible. Stardew Access and Hearthstone Access are two mods currently in development for Stardew Valley and Hearthstone respectively. Blind players can now play the popular farming simulator / social game Stardew Valley and the PC version of one of the most popular card-battling games in Hearthstone.

The mod I’m most excited about though is the Toby Accessibility Mod for Doom. MrAlanD1 has been creating a mod for the original Doom games, including accessible level packs, that make this classic first-person shooter playable by blind gamers. Doom is my favorite game ever, so I’m glad this mod opens up the game to blind players, even 25-plus years later.

Let’s address the elephant in the room. How does someone who is illegally sighted enjoy VR, a visual medium?

Well, unfortunately, virtual reality is not very blind / low vision accessible so far. I saw an Oculus Rift developer kit model a few months before its retail release and was blown away at the level of immersion possible. I knew right away that there were going to be significant challenges in making VR accessible to low vision and blind users, but I wanted to be a part of it. I bought an Oculus Rift when it was released and also picked up an Oculus Quest because stand-alone VR has its own challenges.

One of the most difficult aspects of VR as a low vision user is the in-headset user interface. Menus and other screens don’t have to be restricted to a traditional flat-screen layout. Every experience can, and often does, display things a bit differently. Some games like Job Simulator, use the environment itself as the user interface, while others use a more traditional, flat screen design, with menus and options screens floating off in the distance.

There are currently no commercially available accessibility tools or software that add things like a screen magnifier, screen reader, or high contrast to a VR dashboard or game interface. There was an amazing accessibility suite called SeeingVR, developed as a research project by Microsoft, but it never left the research stage.

While I do have trouble with some aspects of VR gaming, like aiming down the sights of a weapon in a VR shooter, it’s the text and user interfaces that give me the most trouble. Even if a game offers some settings that may help with accessibility, I’m often unable to navigate to these due to largely inaccessible menus.

Tell us about your involvement with XR Access.

When I originally bought into VR and VR accessibility, everyone was still figuring things out. Standards hadn’t yet been fully developed yet, and I was hoping to be able to add accessibility as a core consideration for developers early in VR’s existence. Despite contacting Oculus and other developers, I didn’t make much progress. Around six years later, I haven’t seen much, if any, improvement in VR accessibility.

I discovered XR Access in Spring 2020, and have been a pretty active member ever since. XR Access is an organization devoted to improving the accessibility of both virtual and augmented reality. XR Access has several working groups dedicated to different aspects of accessibility. One group is focused on the business case for XR (bcXR) accessibility, getting support and buy-in from development teams, upper management, and others.

The accessible development (adXR) workstream is more focused on actual development standards and tools that can make XR more accessible. XR Access has recently restructured and is in the process of developing virtual and augmented reality accessibility resources and prototypes that other developers can use when they are trying to figure out how to make their apps more accessible.

XR Access is always looking for more people who are interested in furthering virtual and augmented reality accessibility. If you are a person with a disability and have experience with AR or VR (good or bad), or if you just have an interest in this area, please reach out to XR Access. If you are a developer interested in this area, and making your apps more accessible, you are also welcome. There is no cost to being a member, observing, or participating in meetings, and you can devote as much or as little time as you are able.

I’ve enjoyed my time in XR Access. I’ve assisted with several brainstorming sessions, reference documents, and the yearly XR Access symposiums, taking place every summer.

What should developers do to create accessible VR apps for the blind and low vision?

Wow! This is a rather difficult question as each VR experience is different. I’ll try to focus on a few key things though. First, offering text size options, preferably a text size slider control. This can help make any text in an app more readable, not just for people with significant vision loss, but for people wearing glasses. If an app offered enough text size options, some users may not even need to wear glasses while using a headset, making the experience more comfortable.

Magnification and menu narration features are often critical for blind and low vision users. Some low vision users, like myself, often use a combination of magnification and narration / screen reader. Blind users not only require a screen reader or menu narration but also a control method for locating and navigating VR controls. This is especially the case when using motion controls like Oculus Touch.

Finally, one of the most important features that help with low vision accessibility is the full six degrees of tracking for a headset, from start to close of an app. Three degrees of tracking means you can only look up, down, left, and right. Six degrees of tracking means you can lean or move in any direction as well.

This means a user can look in any direction. They can move or lean in any direction. So, if a low vision user needs to get closer to a user interface element or something in the environment, they can simply lean in or move closer to it, just as they would in real life.

Unfortunately, many VR apps, including VR dashboards, set a fixed distance for much of their user interface. So, if an options screen is meant to look like it’s three feet away, it remains three feet away. If a user tries to get closer, the options screen simply moves farther back by the same amount, maintaining that virtual distance. This is very frustrating, especially when combined with small and stylized text.

For a more detailed explanation of these and other blind / low vision accessibility barriers in VR, please check out my #ID24 2017 session on An Illegally Sighted Look at VR Accessibility. Pretty much everything I talk about there is just as relevant today.

What is an accessibility barrier you would like to see solved?

For both flat screen games and virtual reality, I would love to see platforms themselves include a standard base level of accessibility options for all types of abilities. Both Microsoft and Sony are off to a good start and have accessibility on their Xbox and PlayStation consoles. Both offer basic magnification options and a system-wide screen reader.

Unfortunately, these screen readers aren’t yet compatible with the games themselves, largely due to the game engines like Unreal and Unity, which power today’s games. This is changing though, and both Unreal and Unity are working on more native accessibility features developers can use.

At this time, no VR dashboard has any accessibility options for blind and low vision users. None include magnification or a screen reader. The only way I’ve been able to use as much VR as I have is by experimentation or some other complex workarounds that are too difficult to quickly describe here. Even flat screen interfaces for VR platforms like the Oculus desktop and iOS apps aren’t very accessible. Due to the lack of progress in commercially available accessibility options, I have actually largely stopped using VR in recent months.

There is hope though. More developers are starting to consider accessibility, and organizations like XR Access are doing great work in this area. And when it works, VR has great potential for games, education, socialization, and more.

About Jesse Anderson

By day, Jesse Anderson works as an Assistive Technology Specialist with Minnesota State Services for the Blind. There, he works with high school and college transition-age students. Outside of his day job, he works as a game, technology, and extended reality (XR) accessibility advocate and consultant. He also runs the IllegalySighted YouTube channel and Twitch stream. Jesse is a Low Vision Advisory Board member with Microsoft. He’s involved with XR Access, where he works on improving overall accessibility for augmented and virtual reality.