
Today’s commercial VR systems provide interaction methods allowing people with disabilities to experience gaming and virtual environments as never before. Aaron Gluck discussed developing two VR game prototypes, The Enclosing Dark: A VR Auditory Adventure and Racing in the Dark, neither of which uses visuals to provide information to the player.
He has accomplished this by focusing on the usually supplemental information channels: audio and haptic feedback. Through Aaron’s research and implementations, learn how current VR technology can be leveraged to support highly accessible content.
Aaron Gluck is a third-year Human-Centered Computing Ph.D. student at Clemson University. He works in the Design and Research of In-Vehicle Experiences (DRIVE) Lab, where he researches accessible autonomous vehicle human-machine interfaces for older adults and people with disabilities. Aaron’s personal research interests are related to accessible VR.
For the last three years, he has been trying to answer the question, “What would an accessible virtual reality game for people who are blind look like?” His passion and love for VR have led Aaron to focus on making VR accessible so everyone can share in the experience. Aaron lives in Clemson, South Carolina, with his wife and two daughters.
Here are some of the highlights from this presentation:
- Market opportunity: Video game market
- Research: VR game design for blind people
- Novel human interaction mechanics for driving simulations
- Demo of Racing in the Dark
- Q&A from Aaron’s Talk
Transcript: Virtual Reality in the Dark: VR Development for People Who Are Blind - Aaron Gluck
Aaron Gluck: Like Thomas said, I'm Aaron Gluck. I am a third-year Ph.D. student at Clemson University. I research in the DRIVE lab, and we'll talk about that in a minute.
I did my undergrad at Boise State University in the gaming interactive media, in the media technology department. It was a brand-new major when I attended. And it was there that I was actually introduced to VR the first day of my studies there. And amazingly, I was hooked from the start.
So it's, quote, been all downhill from there. I've been developing VR since 2016. So I've been doing it for about five years now. And my research focus personally is of accessible virtual reality, looking at for people who are blind, and for older adults as well.
That's where I've been doing most of my work. Just a little bit on the DRIVE Lab where I work. It's run by Dr. Julian Brinkley.
His research was in Thomas vehicles, and human machine interfaces for visually impaired individuals. So amazingly enough, our research focus is on automated driving systems. We work on ride sharing, and we also look at accessibility for older adults and people with disabilities in those vehicles. I get to play with fun things as part of my lab, so I have two oculus class 2s, oculus rift and we just got Microsoft, too, which allows me to do multiple research projects here in the lab, working with our participants.
We also have some expensive toys. This is a driving simulator where I actually build simulations for either driving or for riding in vehicles experiences and research. And then our latest over the summer ended up building this. It's actually a full-scale replica of an autonomous vehicle ride sharing experience.
So we're doing participatory design in studies in this vehicle. So a little bit about the motivation for what brought me actually to Accessible VR. My interest grew from my senior undergraduate cap stone project where I worked with a group to develop an application -- a VR application for the World Museum at Boise State University where people would be able to interact with an ancient Aztec codex. We made it available so that those that were unable to use their hands could still use the application.
There's no standing required. And so we were able to bring those things into -- and we obviously had the captions and stuff like that, so making that as accessible at that time. Really my interest crew to impress my soon-to-be adviser, he said, I'm looking for someone that does VR work. I said, hey, I kind of thought a little bit about accessibility, but why don't I try and make this application that uses no visual inputs in VR.
That's actually where the first project we'll talk about here shortly in closing Dark was actually born. My interest has grown in assistive technology class here as a grad student, and obviously by working with people with disabilities and older adults in their lab. So why VR for me. The biggest reason, reason number one, it's commercially available.
There are plenty of studies that have been out there that have been done for people who are blind, and using equipment that costs tens of thousands of dollars. It's just not affordable. So here's a system that's got 3D audio, vibrotactile feedback, and built-in microphone for voice recognition and head tracking built in. I felt that was really the key feature for allowing someone inexpensively to, one, play games, and two, interact and experience things that they just haven't been able to experience previously.
So I just want to -- just some stats on this. There's some kind of interesting things I found over my research and time. In 2019, someone actually estimated that there have been 1.2 million video games released across all platforms. Gamers are averaging about two to four more hours a day of daily game play.
The problem is, there's a negligible amount, a very tiny amount of any form of accessibility features in games, and there's even less for people who are blind. In 2006, though, AudioQuake was actually released as the first adaptation of a mainstream video game for people who are blind. So we're making headway. Just not very fast.
There are over 39 million people world-wide who are blind. That was in 2010. Obviously there's a few more than that now. In 2050, we're actually expecting it to be 61 million people.
That's not a small population at all. Researchers are starting to look at this group. Looking at it from a VR and gaming standpoint. The gamers who are blind, they want to experience VR.
The problem is, they don't really believe it's going to be possible for them to experience it. And I am working hard to make that not come true. And we actually found video games have a lot of benefits, not just wasting time. So you've got the social interaction, so you're improving moods, sense of being, all of these things are important, not just to people who are blind, but anyone who can play in.
And there have been lawsuits now about lack of accessibility. So pretty much the -- at least the first lawsuit that I found about VR was about lack of captions. So Panarra has sued HTC corporation, which is the HTC Live for VR. And they are not expecting the lawsuit to actually go through because they're kind of, in my opinion, they're suing the wrong people, they're suing HTC Live rather than the actual manufacturers of the games.
So it will be interesting to see how this turns out. And then Robles versus Domino's Pizza actually is really making a -- made waves because they -- domino's is ordered to make their website WCAG accessible, or compatible because there was lack of screening access to the website and apps so they couldn't order pizza. So with these things, developers need to really start thinking about accessibility, because we've got, and show that it's wanted, it's needed, and if you're not going to do it, you are going to get sued.
So it leads to some interesting things. And problems. So who's thinking about VR and accessibility. Sadly, very few developers are from this image, this was last fall, for a conference on game developers.
Accessibility is not listed at all in any of those aspects. Some research is finding that there's a lot of confusion about how to design for accessibility, and there's just not guidelines for teaching you how to do it. And then there are some researchers that are doing it. Like I talked about earlier.
As researchers, we're able to afford more equipment and more expensive equipment than you would usually find in someone's home. And so while that research is good, the problem lies is it's not always truly accessible. There are beneficial -- or becoming research labs, so Microsoft Research is one of the big research labs that's researching accessibility. They've done some work with providing for low vision, people with low vision, and they're also doing some research for people with motor disabilities.
So there is some research being done in that way. There are some developers, others trying to do it. Owlchemy Labs, they have added one mode to their vacation simulator, I believe the job simulator as well. They also have non-audio feedback that they're bringing in, so they're trying to make it accessible.
It's kind of piecemeal. But they're one of few that are making that direction. The nice thing is that there are some places that are smashing it when it comes to accessibility, so last of part two has been listed as the most accessible game ever. There are 60 accessibility options and features built into this game that you can turn on or off as needed.
So it is possible. This game proved to me, and I hope it proved to a lot of developers, that it is possible to do this. You just have to put in the time and focus to make it happen. But the reviews have been great from that community.
The people that are definitely thinking, though, about it, are those who are sidelined. Those that are unable to use VR. Many of them are being kept from their desired peer group. They want to be a part of the gaming community, but they're just not able to.
You know, obviously there's concerns of lack of equal access. And one of the important things that I have more and more started taking to heart is this idea of those who are visually impaired who want to not only enjoy the products, but they want to be a part of the development and design processes as well. They want to have their input. All right.
So this finally leads to my research. So my first project I did as I mentioned before is in the Closing Dark. The Closing Dark started from the premise of what would a game look like for a person who is blind. What would that end up being.
How can we do that. What type of things can we do. So for this, when I started, it was choose your own adventure style and game play, you're exploring different mazes. It's very exploration based.
There's some minor interactions. I didn't want to start with anything too complicated, because I wasn't actually sure how I was going to do it at that time. So as I started playing and messing around and exploring, I looked at has become the game play mechanic which is passive seminar. So as you're playing a game and exploring those mazes, going around you is passive center.
So as you get close to a wall, it will ping at you. We'll watch a video briefly about it here in just a few minutes. The passive sonar works actually -- we'll get this in just a second. The other way that we can use this in the game, you actually do have a gun.
There are some things you can shoot if you so choose. You do not have to. But also allows for echolocations, so you can also use the gun to determine distance, the bullet fires as you would expect from a weapon, and then if it hits a wall, it's going to hit that wall after a certain period of time so you can kind of judge where you're at in the environment. One of the biggest things, though, that we quickly found is the idea of cognitive load.
We wanted to avoid that. Cognitive load is how much thinking are you able to do all at the same time before you start losing the ability to process additional information. And we found this actually from our passive sonar system. The passive sonar originally started as a system that will send out a -- basically a ring around you in 360 degrees.
Anything that was hit by that ring, walls, any objects in the way, all at that moment in time pinged when they got hit. The problem became is there were certain times in certain areas where that sonar would tag three to four different objects in the play area, and they were all getting information at the exact same time. So it made it actually extremely difficult to determine where things were in the environment.
So we ended up changing that to a rotary style sonar so it actually still spins 360 degrees around you, but now it's shooting in a straight line. And then when it actually hits something, it stops for a moment, does its ping to let you know where in the environment that hit something. And then it moves on and continues to spin around until it hits something else. That way, people were able to -- our play testers were able to figure out what they were able to pinpoint things in their environment a lot better, and made it a lot easier and not as confusing for what they were trying to do.
So for the Closing Dark, accessibility was really our key in that. Because I wanted to make this for people who are blind. I know there's different levels of blindness, so I wanted to go to the far extreme, knowing that if I went to the far extreme, then everything that was a little less extreme should be covered in that. So I made it with absolutely no visuals.
As you'll see here in just a few minutes, I'm going to show a video of it, it's a black screen. So whether you have no usable vision or you have full usable vision, you see exactly the same thing. There is no difference. So to do that, we really focused on this 3D spatial audio.
That ability for the audio to be able to allow you to pinpoint things in your environment. Once again, it's the ping sensing of a wall, so you kind of know where your walls are, to being able to use the echolocation of the bullets, or if it's the audio of the creatures that are lurking in that darkness. We wanted to be able to allow people to pinpoint those things. So we used spatial audio heavily.
We did use voice recognition for it. So you can ask the system questions rather than having to use a menu. Menu is one of the big things that we worked on. So we do have a voice recognition menu, we also have a manual menu that speaks to you, uses text-to-speech to provide information to the users.
For this, we used a lot of nuanced feedback actually. The feedback system haptics tend to be on-or-off type of situation, where, you know, you're going to feel the full force of that vibration if you are attacking something, or something's attacking you. So we actually explored nuance haptic feedback by looking at how can we, through haptic patterns and different levels of the haptic feedback can we replicate textures. We actually looked at replicating a gentle movement of a butterfly's wings on you in this situation for more of a healing type situation in a game play.
And so we found that nuanced haptic feedback allowed us to do a lot of -- providing you information without, once again, that visual information. The problem with haptic feedback, though, is you definitely need to make sure if you're including it, that you have a training period of time. Hearing a duck quack, you kind of know what that is, if you include that in an environment. Feeling a very gentle vibration at a certain type of pattern is not something that we deal with on a daily basis.
So providing that feedback ahead of time and teaching people how to use it is very important. So those were our takeaways from Closing Dark. That is still in development. I keep making a little bit of headway.
I'm doing it all on my own and trying to do it while doing my Ph.D. it makes for a little bit of a difficult time. So pretty much of Closing Dark trend turned to Racing in the Dark. This is research that I did this last winter break.
And it really stemmed from the question of, in Closing Dark was something that's slow paced. There's no rush. Take your time. And so it's easy to provide that information.
But what happens when you need critical information delivery. What happens when you have a vehicle moving at 100 miles an hour on a track with other opponents that are racing with you. How do you provide that split-second decision-making without that visual input. And how do you provide that continuous navigation along a narrow path.
How do you do that along a track. And so that became the question. I wanted to see how can I provide that critical information. You've got to be able to make a decision what second you're going to crash into another car or into a wall.
And so that was what led to the idea of Racing in the Dark. So for Racing in the Dark, it relies extremely heavily on haptics. We didn't do much with audio in this, even though there's audio obviously. Haptic is the key.
We did do some trials to see how it would work to say, all right, make a right turn now, or a slight left. But there's a lot of problems with that. It takes a little bit of time to say, left turn, and how do you differentiate how much of a left turn that is. Is it a 10-degree left turn, a 35-degree left turn, a 90-degree left turn?
How do you give that information clearly and in the right time with audio. It just didn't work. So we started playing with haptics with this, and that's how it became haptics with the input with this. Modern VR systems all include hand-held controllers with those electromagnetic vibration motors.
While there are different systems out there, and we are moving to hopefully a non-hand-held controllers, there's always going to be vibration somehow. So we knew we could do that, and it would be around for a good long time. We also realized the hand is just a great source for being able to detect vibration frequencies. We are used to touching different things.
And we figure out a lot through just our touch. So using haptics really provided a strong basis for us, for providing that information. And so our haptics actually provides a player with that constant steering information. We wanted to make as realistic of a steering system as possible.
And so you use the controllers, you raise them up and down, and that determines the turning. So vibration is going to let you know if you're approaching a wall, you're drifting out of your lane, or you're entering a turn, you're approaching another racer, all that information that you need to be able to respond to at a split second's notice is provided through the haptics of the controllers. And that haptic information is mostly just through the vibrational amplitude, the strength of the vibration.
So we've got a continuum. Obviously I just list low and high here. But literally we can dial it into whatever it needs to end up being. So a low vibration, just a nice gentle vibration is just a slight turn, or you're getting close to that correct hand position.
And high vibration, you need to crank that wheel as hard as you can in order to make that sharp turn. So we've been able to dial it in that way and allow our racers to make their way around the track. Like I said, at a simulated 100 miles an hour. So we tie the haptics to hand gestures.
Like I said, we replicate the real steering motion, so you're raising your hand -- you're basically turning a steering wheel. You're raising your hand or lowering your hand based on the vibrations. And the system is continuously tracking that vertical distance between the two. And so it knows that if there's almost no difference between the height of the controllers, it's just driving straight in whatever direction you're pulling it in.
And then if you need to make a right turn, your left controller is higher. If you're making a left turn, your right controller is higher. Once again, just like you would in steering a car. And that greater the distance between the two controllers, up to a maximum, determines the sharpness of the turn.
So if you raise that controller up pretty high, you're going to make that sharp, sharp turn. And that's all going to be taught, once again, shown to you, or delivered to you through that vibration. We knew that there wasn't just haptic information that went into it, so we also used head gestures. We wanted to replicate driving a car as much as possible.
So we built in head gestures, because as you drive a car, you do use your head. You look up into your rear-view mirror and we replicated that here. If you raise your head up, verbal information from the game is going to let you know if any of the racers are approaching you from behind, I believe at it's about 100 meters behind you. So if you look up, it will let you know that someone is starting to gain on you.
And then you can turn your head left or right, and that shows looking out your side windows. And so it gives you verbal information again about players to your left or right. We went with verbal information on this, because this wasn't something that you needed to make decisions on in a split second, this is information we knew you could take a couple of seconds to process, and then make decisions on, if you needed to make a decision.
So we took advantage of that. So for accessibility in this, very, very similar to the Closing Dark, once again, no visuals. It is a completely black screen. Which we'll show a little video of that.
I do show some video of what's behind the curtain, as I like to say. But if you're actually playing a game, there is no visuals. But we focused on the haptic feedback. We knew that haptic feedback was going to be our best option for providing that instantaneous feedback.
Then we wanted it to feel like you were driving a car. So we used the gesture tracking, so that hand movement, to give that same feel as a steering wheel, and then the head gestures of looking in the rear-view mirror and looking out the side windows. We do have voice recognition in this. Although we are looking for another system for that.
We use Watson AI, and it's just not as good as we would want it to be. But we do that, so you can ask the system questions as you're driving, once again, for non-important information, or at least not time-important information. We do use text-to-speech, so it is speaking to you, giving you information about where you're at. We do use a little bit of 3D spatial audio, so you can hear the other cars as they approach, or as you pass them.
But it's not nearly as much as in Closing Dark. So this point, actually, we're going to go ahead and watch a little video. It worked out perfectly. It's about a nine-minute video.
I might skip a couple parts, but there's a video screen behind everyone.
Aaron Gluck: It is captioned. A Laughing Lance Studio production.
Welcome to the Enclosing Dark, a VR auditory adventure. My voice is raw from the screams for help of the last few hours, days weeks? I don't know how long it's been since the earth swallowed me and I fell. darkness confudes my mind I know I've awoken multiple times, but whether losing consciousness due to exhaustion, or fear...
I don't know. The only thing I know for sure is I should be dead. There is no way a person can fall for that long and survive. As far as I can tell, I've only sustained a few scrapes and bruises.
[Yawning]. I must have fallen asleep again. But I'm not sure. Eyes opened or closed, the view is all the same.
I know that I slept. And that something woke me. Not a touch, but a noise. Maybe it was just my imagination, or maybe the sounds of a rescuer.
Ow! What is that? A gun? How did this get here?
Maybe I bumped into it while I slept and that was the sound that woke me. No, definitely not! I must escape!
Menu open. Vibration control. Turn vibration on. Vibration is now on.
Current game menu. One remaining.
Aaron Gluck: This is the menu system in the game.
26. Bullets fired. Zero, bullets fired. Enemies killed.
Zero, enemies killed. Quit game. Current game menu. Menu closed.
[Growling].
What is that thing? I've never seen anything like that before. But the truth is, I still haven't. I need to try and determine what the thing was.
I had to. So the last of my courage, I ran my hands over its now lifeless body. I don't know what I was expecting. But it definitely wasn't the warm mucous-covered skin at the back of my throat, as my hands found the gunshot wound that killed the beast?
A demon? A monster? I have no idea. I can't think about it any longer.
I must escape this darkness. I just hope that that aberration was a mistake and that no others exist.
Aaron Gluck: That's the opening of the game. And then we do jump -- there's multiple endings to it. But this is a part of the ending.
If only I had died from the fall. Quick and pain for a moment. The slow, drawn-out death is definitely worse. I feel tears flow from my eyes and know the end is near.
My eyes close again. [Fading voice]. I'm jolted back to consciousness, and something grabs my hand. I struggle.
I realize it also feels like a normal hand, not the gnarled hand of the creature. My hand is squeezed tightly.
I knew I'd find you.
Aaron Gluck: Driving in The Dark provides no visual information. The black screen you see now is exactly what the players see. Spatial audio, haptic feedback, hand and head gestures and voice input. The combination of these interfaces allow players to experience both virtual reality and car racing.
During this demonstration, most of the screen will stay black to remind you of what the player is seeing. Additionally, we will have a video window showing a player in VR playing the game, along with a video window showing what is happening visually in the game behind the curtain. A tutorial begins the game instructing the player how to use all interfaces in preparation for racing around the track.
Welcome to Driving in the Dark. If this is your first time playing, or you want to go through the tutorial again, say learn, otherwise, say start.
Learn.
Aaron Gluck: Here players will learn how to steer the car using haptic feedback and hand movements.
Before you start speeding around the track, let's show you how to drive your new race car. To turn the car, you will raise and lower the controllers much in the same way a real steering wheel works. To turn left, raise the right controller higher than the left controller. To turn right, raise the left controller higher than the right controller.
You will know in which direction to steer the car based on the vibration of the controllers. When a controller vibrates, it is alerting you to raise the controller higher than the non-vibrating controller. Let's try a medium left turn now. The vibrating right controller, lift until the vibration ends.
Well done.
Aaron Gluck: Gain more information of their surroundings with voice commands.
While you're driving, your pit crew will keep you informed of other racers in front of you. If you want to know if any racers are coming up to pass you, just tilt your head back to look up into your rear-view mirror. You can also check to see if there are racers on your left or right by turning your head left and right. Finally, you can verbally ask your pit crew questions.
Aaron Gluck: After the tutorial the race begins. The race is a three-lap race around a 3 1/2-mile-long track with four basic AI opponent racers.
Ready, said, race.
Distance.
You have completed 65% of this lap.
Spatial audio is used to provide the player auditory information about the other racers' positions and the player's pit crew provides verbal information about players in the front.
You are gaining on a car in the center lane. Now in right lane. You are rapidly approaching a car in the center lane. There is a racer directly to your left.
Aaron Gluck: While driving in the dark needs more testing, refinement and content before it will be ready for release to the public, we have shown through interfaces, players with visual impairments can play virtual reality.
Final race time 6 minutes and 18 seconds. Congratulations. You finished in first place.
Aaron Gluck: All right. So that's, I guess I can't point out the exact word on though, showing what we're at for that. So the question becomes, is, where am I going from here. So the main product we're working on right now is we're continuing along with Racing in the Dark.
In the next couple weeks I'll actually be starting to work with a legally blind co-designer. I've finally been able to make some progress into finding a participant to help me with that. So once again we'll start going over design iterations, making sure that everything works as I believe it should. We started this project during the COVID-19 shutdown, so we were not able to use any participants previously.
So it was a lot of closing our eyes, or if we felt like we were cheating, putting something to cover our eyes, to get that same feel, we definitely know that that just gives us a basis point. So we're going to make sure that all the pieces work as we hope they will. Again, with the legally blind co-designer, we hope to finish the game play. We're going to look into that better voice recognition option.
We want more consistency and honestly Watson AI is an expensive option. It costs per, I think about 1,000 words. So we need to deal with that. And we'll do more intense play testing.
We'll get it out to you hopefully more to the community that way. We'll actually get some real voice actors to record. Not just my bad voice acting. Our goal is to make it available to the public.
My hope is going to be probably in the next -- I'm always shooting for sooner than I think it will happen, so I'm saying in the next 18 to 24 months. It probably won't be a full game, but maybe just a couple of tracks. What we want to do with it is make the assets that we are developing for it available to other developers, so that they can bring that into their own racing games, to make it more accessible.
And then I want to explore first-person shooter games, see how that would work for making something for people who are visually impaired. So I figure that's kind of the next, you know -- Racing in the Dark was the level before Closing in the Dark. And the shooting would be the next level from there. So to wrap up, what I hope to show from my research, and what's really inspired me for it is, I believe it's possible to create VR games.
And I feel like I've shown that it's possible to create VR games that require no visual information. They're playable and usable. And so while more accessible hardware would be beneficial, we also wanted to show that it is possible to use it just off the shelf, commercially available VR systems for accessibility for people who are blind. And really, now is the time for developers to embrace that accessible VR application development, before the policy states that it has to happen.
Start looking at it now. It just takes time, energy, and being willing to kind of think outside the box of how do we provide that information. Because I'm a Ph.D. student, I have to have references, it's just built into my DNA now.
So here's some references for papers that I've talked about in this. I just really want to say thank you to Thomas and Meryl for reaching out to me to do this talk. I was very humbled by that. I also want to thank everyone who's been working behind the scenes to make this happen and getting this streamed onto YouTube.
So I think we've -- at least on my clock, I ended one minute before that warning you were going to give.
Precise. What a precise presenter. Thank you so much, Aaron. I'm going to hit the 3 button on my computer to give you a round of applause.
If you want to give a clap for Aaron, you can hit 3, or you can use the react button. That was an awesome presentation. I definitely saw activity on Twitter, too. You have some exciting and interesting mechanics that you put in there.
I learned a lot. I'm going to take questions from the YouTube stream, and then we'll open it up here in the room. We had a question streamed from Deb. Should Closing Dark should be used in a lab.
Do you have a thought if that can be used as an experience for helping people build input for people who are blind?
Aaron Gluck: It's a tricky question. I have friends who are doing empathy work. It wasn't at least what it was designed for. It would definitely -- I could imagine it being a by-product of playing it.
But I think the idea more -- one of the things that I've heard said a lot is that, VR is used to provide empathy about people who have disabilities and things like that. And the problem is, while that's a great thing, the problem is, it's not being built for them to use. And so while yes I think this could be used for that, I really want to focus on the aspect of providing an option for people who are blind, to actually play and experience.
Thomas Logan: Absolutely. Thank you. We've got a question from Cody Fitch. Do you feel the spatial audio is good enough in current headsets to create a good simulation of a physical space?
And the nuances that go with it? Or do you feel technology still has a ways to go?
Aaron Gluck: I would say that -- from a basic standpoint, I think it works great. You need to be paying attention if you're using it to not put too much ambient noise in there, that's going to interfere with what you're trying to do from a 3D audio standpoint for interacting with the environment, or anything like that. So where it has, I think, the most room to grow is from that standpoint of being able to handle a lot of audio input in using that 3D spatial.
But if you are cognizant of it, and you're thinking about it, then it does work. Obviously there's always room for improvement.
Thomas Logan: Great. We have some more questions from YouTube, but I'll also open it up here in the room. Is there anyone here in the room that has a question? Feel free to go off of mute or type into the chat.
We'll just take a moment here.
Dylan: Aaron, this is Dylan. Can you hear me?
Aaron Gluck: Yeah. Hey, Dylan.
Dylan: Thanks for the great talk. I'm wondering, for a lot of the design techniques that you shared here, like using haptic feedback when you're approaching a turn, how many of those do you think would be possible to implement as accessibility features in a mainstream racing game that does have visuals? And how many do you think would only work in a no-visual scenario?
Aaron Gluck: Sure, that's a great question. I've actually been curious about that. One of my hopes with it is to actually be able to get to the point where you could add this to a racing game with visuals, so that everyone could play at the exact same time and you're racing against everyone else. The trick with it becomes that you do have the -- the haptics are running a lot of kind of complications in the back.
And you're having to track the players' speed, the distance to whatever obstacle it is. So definitely it needs more testing to do with that. But my hope would be that it would be possible to make just one full game where everyone could then play.
Dylan: Awesome. Thank you.
Aaron Gluck: Yep.
Thomas Logan: Cool. We've got a question here in the chat. Love to hear your thoughts on developing accessible VR interfaces which replicate real world interfaces, for example, a mirror in the car, versus those that are virtual, i.e., the sonar in the first game.
Aaron Gluck: Sure. What I've played with and experimented with, and what we're looking at, we're really trying to find what's going to fit the situation. So replicating the real world environment, we wanted to give that feel of driving a car. We definitely didn't want this to let anybody think that would be legally blind, by playing this, you would actually get good at driving a real car.
We hope that no one would attempt that. But we really wanted to replicate that feeling. In this situation, because it was such a specific environment that we were trying to replicate. As for using the sonar in the first one, the sonar really is just kind of to a degree a version of echolocation, and actually visually impaired individuals can learn to do that for themselves.
They can snap, and then listen to the echolocation feedback around them. So kind of like a sonar system. We didn't want people to have to click as they went through. So I think really the application of the accessibility kind of needs to fit the environment that you're building.
And does it make sense.
Thomas Logan: Thank you. Does anyone else in the room have a question you'd like to ask? We'll take this one. It came in from HARV.
Aaron, you mentioned the cost of IBM Watson. For synced speech, can you compare it against Amazon or Google's or any others?
Aaron Gluck: I haven't yet. My biggest concern is anybody that -- obviously I'm on a major shoestring budget. And so anything that's going to charge, in my opinion, per X amount of words, a thousand words, a thousand recognitions, whatever it is, is going to get costly over time, especially if it's actually released to the public. So I know there are other options out there.
Like you said Amazon and Google. I didn't compare them. I went with Watson because I've used Watson before. And so I knew how to set it up.
I was in a time crunch to get this created, do some initial testing and actually get a paper written on it. So it would be good to look at other options. That's part of what we're going to be doing is find out which the best option is for us going forward in this situation.
Thomas Logan: And we've got a question here from Meryl Evans. You mentioned more accessible hardware would be beneficial. What would you say were some features, or just like a commercially available hardware that should be considered?
Aaron Gluck: Sure. This kind of actually ties into research that I'm doing right now with VR in older adults. So simplicity is their buzz word when it comes to the equipment and making sure that it's accessible for them. We had individuals that had hearing implants, not cochlear, but cochlear style implants, that went straight across so they could not use the head strap.
And their hearing device at the same time. So those are important things. The other things is the number of buttons on the controller. I think to a degree, the ten buttons on an oculus quest and two joy sticks is just a layover from what has just become the way controllers are made.
But I did find -- I can't remember what system it is, there is a new VR system that actually has 12 buttons. I think that to a degree is getting a little overkill for some people. So being able to just provide what's needed. There's a lot of ways to do input in a virtual environment that I think simplification is going to be a buzz word to a degree for making those devices more accessible.
Thomas Logan: All right. Thank you. We're going to take one last question. The last question of the night, or the day.
Have you explored the use of glasses, no visuals at all, just audio?
Aaron Gluck: I haven't used those just because to a degree, I mean, that's what the VR headset is. Or I can make that the VR headset. I do know they're out there. I haven't looked into what type of controls you can attach with them.
The reason for VR specifically is I had all those specific built-in features. You know, obviously the 3D audio, spatial audio is there, the gesture tracking, all the different inputs, the voice recognition, all that's just right there with the VR headset. So it would be an interesting idea to look at. But I definitely want to make sure that the -- it would be easy enough to support game play.
Thomas Logan: All right. Well, thank you again so much, Aaron, for the presentation tonight. Seeing how the technology works, the slides, the captions and everything worked great. The video also came through.
Great stuff. Thank you.
Aaron Gluck: Thank you again so much for having me. Thank you all for coming. I really appreciate it. And especially thank you to -- I know there were quite a few people from Boise State that at least made it to the YouTube one.
So I just wanted to say thank you all for coming out and supporting.