Adapting VR Games for People with Disabilities | Accessibility VR Meetup Recap

A11yVR with Greg Bednarski

Beat Saber, Job Simulator, AudioShield, BoxVR, and many more games are engaging and very immersive. But they haven’t been accessible for everyone… until now. Players with disabilities can play the best VR games with WalkinVR Driver.

Greg Bednarski presented adaptation methods that make virtual reality more accessible.

He demonstrated adaptation methods provided via WalkinVR that allow people with disabilities to play in virtual reality:

  • Play with another person’s assistance
  • Allow people with limited motion to move with controllers
  • Use controllers to move virtually (also called space drag)
  • Engage without using controllers

Greg Bednarski is a big fan of VR who has a mission to help everyone to join the virtual world, regardless of their disabilities. He designed and created WalkinVR, the first software that adapts virtual reality for people with disabilities. VR opens a world without boundaries, where physical limitations don’t stop them from discovering all that VR has to offer.

Here are some of the highlights from this presentation:

Here’s the transcript [PDF].

Transcript: Adapting VR Games for People with Disabilities - Greg Bednarski

Greg: So today I will be talking about making virtual reality accessible. What usually comes with virtual reality? First of all there is headset, which is basically obvious for everyone. Secondly in the hands we are holding the controllers and when we move them we actually see that something is moving in VR.

So we need to move controllers in order to interact. And additionally on the controllers there are buttons, so we need to press them to interact with some objects in VR. So basically this is in order to move in virtual reality we need to move in the real life. And what is the idea about the WalkinVR project?

What problems we may have? First is inability to hold the controllers, which can be the case when we have spastic hands or after a stroke or after some accident, tetraplegia. We may have movement limitations. So for example, we may be able to operate with hands, but only to the level of waist, for example?

So, this is with muscle atrophy, muscular dystrophy, or after stroke as well. And problems with using controllers, so we are not able to operate with buttons, and it happens also with many places. And the last thing is that we cannot keep the vertical position and we must be in the reclined position. This presentation will be a little bit mixed.

Before I was talking and showing slides but with WalkinVR I created some videos and I found that they are actually the best to explain what it really is. Let me start.

VIDEO: Virtual reality can be an amazing thing. It gives people the opportunity to experience things. They wouldn't otherwise be able to like flying or traveling through space as well as more down to earth activities, such as just getting a good workout. It's not quite so easy for everyone though, with virtual reality or VR real-world movements to control your actions in the virtual world, this is a big part of what makes it so immersive.

But what happens when your real-world movements are limited as a result of disability? It means some VR games and experiences are simply not an option for you. For example, a game may require you to turn around, but if you use a wheelchair, that's not so easy, especially if you're holding the controllers And some disabilities mean you can't grip things firmly, so you may not be able to hold the controllers at all, or you might have problems using the button.

Thankfully there's a solution. WalkinVR is software that allows you to adapt a VR environment. So that limitations in your movement don't have to meet limitations in gameplay it works on the steam platform and integrates with the steam VR environment, meaning it can be used successfully with almost any game. Let's look at four ways.

WalkinVR can adapt an existing virtual reality environment to make it more usable by people with disabilities. The first is a feature called virtual movement. This is helpful for when you can't walk, but you can move your hands and hold a controller. To use it, you simply grab space with a button and then move or turn around.

The second is controller position adjustment. With a condition like muscular dystrophy, for example, you might be able to hold the controllers, but not maintain them at the height required by the game. Position adaptation allows you to adjust the position in which the controllers are visible in the VR environment.

Greg: I will stop for a moment. On the left side -- this is maybe not the most beautiful user interface, but you can see the player, and this I think it's yellow, but I could be wrong, are the controllers, and the red ones are where exactly he or she can see the controllers. So, we can basically adjust position and rotation of controllers in arbitrary way. And actually this is very simple but very powerful, the idea that makes sometimes difference between allowing someone to play VR or not.

Even, for example, in the example on the game on the right side. If you can imagine that you cannot bring this -- I'm not sure what is the English word for that -- to the level of eye and aim, because you just are not able to bring your hands up. Then it's quite natural that this is a problem, and this solution I've seen in many, many cases, this is helping to achieve that.

VIDEO: In the simplest case this means showing the controllers higher in VR than they are in reality. Where there are more advanced options too, like using rotation, or the distance the controllers are from your body. The boost part of this feature amplifies your physical gestures. This turns small movements in the real world into much larger ones in the VR environment, making it easier to pick up or throw virtual items.

This can be particularly helpful if you have a limited range of hand or arm movements. Oh, and you can also adjust a controller position for each hand individually. This can be helpful where one hand or arm is weaker than the other. For those that have had a stroke, say.

It means you can compensate for the weaker arm, enabling you to bring the virtual hand up into your field of view. This is something that can have a positive psychological effect, and there are more parameters that you can adjust, besides. The third feature is gameplay with personal assistant. Some people can move the controller as well enough, but have problems with buttons and other tactile aspects of gameplay.

Gameplay with personal assistant allows another person to assist the disabled player, using a separate game controller. They can make helpful adjustments to the VR position, and also perform button presses, making gameplay easier for the disabled user.

Greg: So you can think about this like co-play or like the name applies, game play with assistance. As you can see the therapist is holding the controller, he's using the joysticks and buttons to, let's say, help the person playing VR interact with the environment. And I must say that, for example, then you can have kind of playing together, and it's quite natural because then you start talking, okay, let's go here, can you grab this?

Okay, I have this. Now let's move there. And this is basically how it works.

VIDEO: Finally, there is hand tracking. Many people have problems holding controllers. Those with some forms of cerebral palsy, for example. You could attach the controllers to yourself in some way.

but this can be uncomfortable, and they can be easily dropped. By using WalkinVR, in conjunction with Kinect, and potentially other tracking devices, you can interact with VR directly, using your hands as controllers. Indeed, many games and applications can be played this way, including workout routines. Put together, all these features mean you can finally focus on the whole point of the exercise playing the game.

We're so excited to be able to open these possibilities up to a wider audience.

Greg: Okay. And I will show also a short game play with Beat Saber. I'm not sure if you know Beat Saber, but this is the number one for virtual reality, and I was once on a visit with my friends and we actually had been playing this game. So, let's see how it works.

[VIDEO DEMO]

Greg: Like you can see, this previous player, you might think that he would have problems playing VR, but if you look on his result, he did a combo with 24 hits in a row. What does it mean? It means that he was able to cut the box in the game 24 times without any failure. So it shows again that VR can be really played by anyone.

So, what exactly is WalkinVR offering? It's an ability to play commercial games, that means games on the market, right? And what I have here, if someone knows VR you probably heard about these games, Job Simulator, Angry Birds, Fruit Ninja, Audioshield, BoxVR, Aquila Bird Simulator. And the difference between the rehabilitation games, is the rehabilitation games perform a specific function, and they are often very simple, but commercial games are very addictive, they are engaging, they are a break from the reality, they are interesting, and there are different types of games, music, training or arcades.

Technically, what is WalkinVR? It's a software for PC. If you're going to ask, can it work with a Quest, the answer is not, unfortunately. But with Quest, you can connect it to PC, and use it with PC, or you can stream it over Wi-Fi.

So, this is software and, apart from the VR setup, we may need an Xbox controller for game play assistant, or Kinect for hand tracking. I'm also working to support other devices like Intel RealSense or (?), which are similar devices, very similar to Kinect. We have some collaborations, and this is one with the university of Alabama in Birmingham, and there has been a recently published paper. I will not go through this paper, but there are some slides from WalkinVR showing how it's used, and we see that they were indeed applying this in the hospital.

I think in the results we can find some interesting information. Game play appeared to improve participants affect and alertness, and motivate them to be more engaged in mobilization therapy. Which is a great thing, I think. I have one more movie to show you, and this is the case study from the very first customer: We started in 2018 and they use it so far.

Yeah, let's see.

VIDEO: Avalon Active, a division of the Avalon Foundation, has Virtual Reality in its offer since October 2018. Thanks to the VR, Our beneficients have fun playing games, at the same time doing exercises that the therapist shrewdly throws in. They do specific movements, have to increase their reach. Prompted by the game, they use various movements and postures, to reach out, grab something, etc., while at the same time the therapist is doing his job, provoking certain movements, so that the virtual exercise could match the real life needs.

It's a kind of enhancement. WalkinVR is a necessary addition for a therapy exercised in virtual reality when we want to address precise needs. Not all handicaps of our patients allow for free roaming the virtual world within its standard parameters. Some dysfunctions require dedicated adaptations, and WalkinVR makes it possible.

Tweaking some parameters we can already in this initial phase adjust the reality of the game e.g. lift a controller higher, turn it around, shift in space, or increase its resolution, so that patients with considerable handicaps can have their virtual training, because the game was adjusted with WalkinVR, allowing for individual adjustments for each person. Most often we use the Box VR game, Audioshield, Beat Saber, Funhouse VR, Run Away VR, these are the most popular games.

In addition we use a game by WalkinVR, dedicated for seriously handicapped persons who have no voluntary, intentional movements. We use Virtual Reality to treat practically all of our patients with all kinds of problems and diagnoses, of both orthopedical and neurological nature. These are multiple sclerosis, cerbral palsy, spinal cord, and traumatic brain injuries, musuclar dystrophies, spinal muscular atrophies, neuropathy, Lyme disease, and other kinds of orthopedic problems. We've noted that patients using the VR get bolder in their physical activities when training in the VR, and this continues on to the real world, as attested in actual treatment, as if the virtual reality exercises top up and augment th effects of the classic, functional physiotherapy.

Avalon Active primary interest are people with impaired mobility, over 16 years old. Our activitu is co-financed by the public and private institutions. we aim to increase the everydayu independence of physically handicapped persons.

Greg: Actually what we've seen in this video, it was kind of accessibility room we can say. There is a room dedicated to VR, it can be also the space in your facility. I've seen also some customers, they have like mobile VR. There is a big monitor, Kinect on the top of that, and there is some part of it with a PC and VR, and they can move it around the room.

This is also possible. This is actually related to how WalkinVR have business model, basically the application is free on Steam. It's free, but there is some banner every 10 minutes, and you have to watch the banner for a few seconds, but it doesn't remove any features, after some three hour period. And then you can buy personal license to remove the banner.

This is for personal users. For commercial users, there are some additional features which we are working, like individual profiles that you can, for example, save some settings for a particular player or additional cameras, which I mentioned, and it's based on the monthly subscription. There is a monthly fee for using WalkinVR for other reasons, or for commercial or non-commercial, for some research, other stuff like that. In summary, with WalkinVR we have gamification of the activities.

Like you can see on this video, it was not that the therapist just give a game to that person and say, okay, enjoy the game. I will sit and look at how you're playing. They were very actively participating in the game. They are trying to use VR, and use these games, which are not terrible games, but they try to use it for therapy.

Because therapy you can do even with a ball, so what is the difference if this is a ball in VR or a ball in the real life? Virtual reality accessibility by WalkinVR. We also have a very important mission, to make virtual reality accessible. I must say it, and -- I don't know, I'm proud of it, but I would say it's very important to know that WalkinVR is a software that is used by people around the world.

Once there was some problem with WalkinVR and I got -- actually it is funny, because then I got feedback from people who were writing me that something happened to WalkinVR, and we need this to use it on a daily basis. So, it's very important that software like this exists. Finally, VR in accessibility room. I would like to encourage everyone to have such a room in your facility.

It can be in addition to VR like on this case study. I'm not sure if this was in video, but generally if patient is supposed to use VR, they have like three or two, or sometimes four, regular sessions of therapy, and then they make one session of the VR. So this is something that is supposed to make some fun and engagement to ongoing therapy sessions. We have to remember that, for many people, they have therapy for their whole life.

So, this is very important to have something. And that's it basically. Before I ask for questions, if you want to get in touch with me, if you would like to have such accessibility room or you would like to consult for your case to use VR in some specific case, you can write to me on this email [[email protected]], or you can go to the website and there is contact form, and you can contact me this way.

Thomas: Greg. Thank you so much. That was awesome. I liked the way you were using the video so we could see real demonstrations of what your technology does.

As I said at the start, that's really what drew me to your project was the fact that you kind of clearly showed in the videos that you showed in today's presentation as well that you are working with people with disabilities and they're immediately benefiting from the technology that you're building. Greg, I'm curious, what would be the next feature you would want to implement. or what's your next idea that you're excited about, that you would want to put into WalkinVR?

Greg: Right now I'm working on integration with other cameras. Like I said, RealSense camera and also there are a few other Kinect alternatives. The feature that was recently added is recline game play. Actually it's simple story for that, because there were two people that contacted me after release on Steam and they say Hi, I have someone, or I'm in the reclined position, can you help to do something?

And I said okay, yeah, you really need that, right? So, I added this ability to play in the reclined position. Later, I realized on some meetings that there are more people using it. So actually it was very, let's say, great thing to hear.

I don't have any document for that, but I have this photo.The person is looking up physically, but in VR you have preview of his eyes and he's looking forward, and his hands also are directing forward. So, this is kind of tricky. You could imagine this in real life if you have a window in 45-degree angle. So, this was also one feature added recently.

I was thinking also about -- but I don't have very high demand from this, but I was thinking about people who have very weak hands like with spinal muscular atrophy, not people who can hold controllers even though they can't bring them high, but there are some people with very weak hands, and I was wondering if we could use device, like Leap Motion, to emulate the controller position. Anyway, I must say that the software like WalkinVR will never be complete, and say it's over, but I have to say that all these features that you've seen, they come from experience.

Each of the features was created because someone said, Okay, Greg, I cannot play VR, or I was trying to get someone to play VR and he was not able to, and then I added something. When you are using these features, they basically make a quite good set of tools that can adapt virtual reality to many, many, many places.

Thomas: Great, thank you. Your next question is from Daniel. He said: Amazing. I'm relatively new to VR.

He said are their voice commands that can be used to navigate the VR space?

Greg: Yeah, that was also one of the ideas. So, we have to remember where we are, if we want to do a VR experience, then there are software that is doing voice recognitions from IBM or Google and we can use it very easily and create our experience, but if we look at what WalkinVR is now, it's a software that is adapting existing games to VR, and we have to think how we can emulate controller and movement and buttons with the voice.

I think there is some possibilities for that, but I haven't heard a lot of questions about it. It's definitely possible in many aspects, but I haven't got such inquiries. Right now I'm focusing on making VR more user friendly to make a new website with better descriptions, and by the end in Q2 of next year I also want to make a better instructions video. Now I'm showing mostly the concepts on YouTube, for example, and WalkinVR is -- it's like I said, there are four or five tools, they are not just buttons that you press, they are tools that you can use with the patients.

Basically my focus now is making the community, everyone, more aware that there's WalkinVR, you can use it, how you can use it. Maybe one more thing to add is I often find that people who use it have some experience in the past and they now see okay, I had this experience, now I see how it works, and yes, this is it. So, I want to be more clear why I created this, and how it works.

Thomas: Thanks, Greg. Next question from David. Great presentation. Looking forward to exploring what your software can achieve.

Are there any plans for porting the software to wireless options like the Quest, or is this too limited at this stage?

Greg: The problem with Quest is that it's closed environment. I'm a bit afraid of the fact that we have metaverse, and we are basically limited to only one company. So unfortunately on Quest or other platforms like that, it is not possible at the moment, but I have to say a good word about the Oculus, they added recently some requirements for each game -- before they allow the game in the store, it should comply with certain accessibility criteria.

For example, I remember one, especially, which is that you shouldn't be forced to lean down to the ground in order to pick up things. You should be able to pick up things from the ground in a seated position, or that you should be able to rotate like with just controller. Snap-turn. So you snap the finger to the right on the controller, and it turns you by 30 degrees or 45 degrees.

They added definitely some features, but probably not as advanced as in WalkinVR. But I would like to say that we shouldn't be afraid of using PC and VR in case of accessibility. I'm sure one day everything will be very simple and easy, but now if we have the way that is doing this -- I'm not just say WalkinVR, then use something else and we can achieve this goal, then I think we should use it, and maybe in five years or something like this there would be something else.

But right now we have this and we shouldn't look for -- yeah, that's my answer.

Thomas: We're going to take a question from YouTube now. This question is from Heather Dodds. Relating to spatial sound and even things like smellovision, hot areas or expanded areas of VR research, has WalkinVR thought about working in these new areas?

Greg: The short answer is no. I haven't tried anything. I was focusing only as you can see on the mobility, this is mostly where VR is now. Also for the commercial applications and for the mass market, because WalkinVR is aiming to make commercial games, or existing games available, and focusing only on what these games are currently delivering for people without any limitations.

Because there is nothing else right now on the market, I was not really thinking about this. This is maybe more left to some research or university work. I just want people to play the games.

Thomas: That's something that I thought was so awesome about your application, is that it works with existing applications, and it's not like oh, only this application or that application. In theory your application can make the most popular applications that don't have these considerations work for people. We have the next question from Randy. this is about your commercial version.

How many player profiles can you save, individual profiles?

Greg: I was not really thinking about this, but a lot. You can save as much as you want, unless you think about some very big crazy number, but I don't think it's the case.

Thomas: Next question from Rem Tyler. Great work, Greg, sounds like WalkinVR is used to modify existing products. Is it possible to incorporate WalkinVR into VR games and environments themselves? We are building a VR platform at work and I want it to be as accessible as possible.

Greg: Okay. So I want to clarify. We are not modifying the games, WalkinVR is modifying VR environment. This is very tricky difference, but if someone was playing VR, then you know that before you launch, when you put headset on, you can be in VR environment, and not even launch a game.

Like now, I have PC turned on, but I haven't started let's say Microsoft Word. What WalkinVR is doing is it's changing the platform, it's working from the platform level, not on the game level. So for the game, the game think that the hand is up, but it changes it on the level of platform. And if you want to create an environment with such features, then basically you don't need to worry about it, you do it like you would do it without thinking about accessibility concerns, because the WalkinVR is already making the VR environment accessible for you.

I'm not sure if I explained this correctly.

Thomas: That's great. There is another question from Heather Burns, the link to the research study or university studies. in the follow-up to the meeting we can get a list of links that people can read or review with what's been done with WalkinVR. We can share that out to everyone.

And also to people who signed up that weren't able to attend today.

Greg: There is not much at the moment, but I can definitely share.

Thomas: Great. Well, I'm going to say again, thank you so much, Greg. We're honored to have you here today. As mentioned, your work has already been presented at our meet-up multiple times.

Greg: It was very nice to hear that.

Thomas: we're very honored to have you here today with us. Thank you so much.

Greg: And I encourage everyone to contact me for VR accessibility room.

Virtual Reality (VR) Accessibility Consulting Services

Our years of experience working with virtual reality and being speakers on the topic have given us a unique perspective when it comes to consulting on VR projects. If you’d like to innovate in the accessibility of VR please, please contact us to discuss how we can help you.