Accessibility Virtual Reality: User Studies with Children and Seniors Presentation

Today I would like to talk about our VR research project with seniors and kids. My name is Kenji and I'm working at Equal Entry as an accessibility consultant. I was born and grew up in Japan, and I'm currently based in Kyoto, Japan. I'd like to thank everyone involved to set up this meetup and give me this opportunity to talk about the Equal Entry's VR projects.

And actually, I have given a similar presentation in Japanese at A11y Tokyo event in January. It was an online event, but we are trying to do hybrid events in Tokyo. if you're interested please sign up for it. We have a Japanese page in Connpass and English page in Meetup.

So let's get into the main topic, VR research with seniors and kids. This is the picture of us testing VR with a pair of senior and a child by side. And in this project we worked with two other organizations, the icon on the left says Azalee Group, and we did this test in their facility. And the icon on the right is VRTogether.

So we tested to see if VRTogether software improves the lives of seniors, and also the education of young children at Asalee. Richard is the CEO of VRTogether, and I remember that he actually presented here at A11yVR before, and they are creating a virtual reality social game platform, and VRTogether is designed to combat loneliness and improve the wellbeing of everyone, and I think he's here today, so if you have any specific questions about VRTogether, I think he can probably answer them.

Also if you are more curious, I'll explain about what VRTogether is, but if you're more curious, I think you can go to YouTube and watch the past A11yVR recording of his talk. And here's the picture of members of Azalee Group and members from Equal Entry. Azalee Group in Tokyo has a senior home and nursery school and rehabilitation center, and they are working on how to improve the quality of life of senior residents and also provide better education for young kids.

At the front row of the image, there's me and Josh, an Azalee representative. I think he's also here today, so if you have any specific questions about what they do he might be able to answer them. Richard is here, and also Josh is here. Thank you very much.

Richard has already explained in the previous meetup, but let me explain again what VRTogether is and what you can do in it. So there are many different fun activities in VRTogether, like on the right there is a picture of a beach scene. There are four chairs surrounding two tables. And, on the table there are like blocks with alphabet on each side, and there's a bucket on the ground, and people really like throwing the blocks into the bucket.

You can also play a Dot2Dot game in a forest, and you can also fly over the Swiss Alps. Let me explain the beach one first. We used the beach theme to let the participants practice the basic controls in the environment, and users can grab the blocks on the table by using the trigger button, and also users can virtually shake hands with each other. That's also a first challenge for them.

And, in the forest scene, there is a Dot2Dot game where players can connect the dots. It was great because it did not require any buttons, so it was a little easier than grabbing the blocks or shaking hands. We have a similar game in Japan, so I was surprised that this connecting dots game was a universal thing.

And so, after connecting dots, a camping tools appears such as like this tent. Players can collect camping goods by keep connecting the dots floating in the air, the more they connect the dots, the more camping goods populate the environment.

The third environment we chose for this testing was the Swiss Alps, and users can fly, ride on a hot air balloon together, and fly over the Swiss Alps, and also they can use their virtual camera to take selfies. That was the explanation for the NPR together. Again, if you're more curious about what you can do in the VRTogether, you can look at the past A11yVR presentation. So, who are the participants?

Who did we test with? So, seniors from ages 70 to 80, and they go to a day service at Azalee Rehabilitation Club, to avoid staying at home all day and to prevent the declining health condition, and they do various activities including physical rehabilitation and socializing with others. And kids, the age were around 5 or 6 and they go to Azalee Pre-school. They offer daycare services for children from zero to 6 years old in the same facility as Azalee Rehabiliation Club, and they teach programming and science and technology and STEM education.

They also learn to communicate in English through games and physical exercises, which is a little unique in Japan to educate kids in English from early age. I wish I had the education when I was little. We gathered these participants to evaluate if VRTogether can help improve the quality of life of seniors, and improve the education of kids. This was actually a pilot testing before doing the larger scale testing.

We wanted to do a pilot testing by ourselves before actually getting funded into a larger scale testing. The purpose of doing this pilot testing was first to evaluate the feasibility of VRTogether as a tool to improve Azalee residents quality of life. A second reason is to create a repeatable process to do VR user testing. And third, we wanted to verify if seniors and kids can actually use VR.

In this image, for example, the hand controllers seem a little too big for kids, and we want to make sure if they can actually use them.

Also, we wanted to prepare to get more funding for larger scale research, and also, most importantly, share the findings with you guys through this meetup, and also the article we published on Equal Entry's website. The whole process of testing was like this, we did a small survey before and after the VR experience, and before they get into VR we gave a controller instruction because VR controllers can be very difficult to use.

So, the process was, we did a small survey first, and then, before wearing a headset, we explained how to use the controllers, and then we did the main VR experience, and then we did the after survey. Because it was hard to ask them to write, the Azalee staff asked the kids and seniors prepared questions and they took notes. Because this was a pilot testing, we kept the questions very very simple and minimal.

For example, this is the before survey, but VRTogether provided us with their past survey to evaluate the mood change of the participants. On the right, that's the original survey, it has so many questions.

But, in this pilot testing, we simplified down to just two questions for the before survey, and we translated those two questions into Japanese. First question was, How do you feel at the moment? And the second question was, How much do you know about VR? And same for after survey.

This is the survey we did after the VR experience. On the right, again, there's a survey with a lot of questions that VRTogether used before in the States, but we only used three questions, and two free texts, for this pilot testing, and first question again was, how are you feeling right now? And second was, How was the experience? And the third question was, Do you feel like you want to do this again?

And the remaining two free texts was, What was good or bad about the experience. Once we get the proper funding, we're planning to do many more questions to get more precise data for this test, but for this one, we kept it very simple. Because we asked how they feel before and after the VR experience, we got to know how much VR influenced their mood, roughly.

This is the graph. This graph shows the mood change of each participant very roughly. So, again, as this was a pilot, we only had four participants for this chart. It says one 82 year old participant did not feel any mood change, and they stayed at 1 out of the degree of 10, but one 79 year old participant experienced a huge improvement from 4 to 10, and two other participants at the age of 5 and 6, they stayed at 10 before and after the testing.

They were very very energetic. So, the only question was how are you feeling? This may not be enough at all as an academic quantitative research yet, but this was mainly to verify the whole testing process, and the logistics of inviting the participants and doing the survey and testing controllers and doing the VR and doing the after survey.

We are hoping that next time, if we do the larger scale research, we can get more meaningful quantitative results with a larger number of participants. After the before survey we did, we explained how to use the controller. As it turned out controllers are very difficult, and if you are not familiar with games, it can be very difficult. For example, this senior person on the right tried to put his arm through the controller's ring, because Oculus Quest 2 controller has a big ring on each controller, and it was very hard to tell the concept of what the controller is and what it does, and this person tried to shake hands with another person in the virtual environment.

Normally you have to keep holding a controller and get closer to the other virtual hand. But, when we said, Could you try to shake hands with the kid? The person instinctively removed the controller and tried to use the real hands to shake hands, and it seemed that it was difficult for her to understand why we need to use the controller. And also buttons were very hard as well.

Sometimes you have to press once and release, and sometimes you have to press and hold, It was difficult to differentiate those actions.

And headsets are still very heavy. Sure, it has improved from the past devices, but this person is putting his hand on his face and showing it was a little exhausting, and we used a head strap that lets you use the dial to stabilize from the back, and it seemed to help a lot, but still heavy. When I had the chance to try Quest 2 Pro headset, in this test we used Quest 2 headset, but when I tried Quest Pro, the latest one, it felt much lighter, but weight and stability are still issues here.

Although we have found many usability issues, we also had very good reaction as well. When they first saw each other in the VR they seemed very excited. The child instantly shouted, Who is it? And the senior responded, It's me!

And they seemed to enjoy the moment, and we could see that it really brightened their mood, and it actually improved my mood as well. It was a very fun moment.

And but the problem was kids learn fast, but it was often difficult for seniors to get the hang of it. The problem was, because kids can learn fast they start playing on their own, and when seniors are struggling, there was no interaction between them, and it was just staffs trying to teach them how to use their controllers. I feel like it would've been better if we had like a tutorial session that requires a collaboration, and also, once they get in VR, it seemed like everything was very confusing for them.

Even what we think is the simplest instruction was sometimes hard to understand in VR. In this case, I tried to tell the participant to extend their arm. She seemed very confused and stayed still. She might have had a physical limitation and condition, but it looked more like, because she was in VR, even extending arm was very hard, and she was a little at a loss.

This is just my guess and we need to verify, but it is possible that VR, because it's all new and scary to them a little bit, some people are hesitant to make physical move at all, such as extending the arm, or even looking around and rotating their head. From kids, we generally had very good reactions. There were some quotes from them, and one said connecting the dots game was fun, and some said they want to throw more blocks, dices, and the most excited child said that he really enjoyed punching the other avatar.

Obviously there's no punching feature, but he was like pretending to punching, he was really enjoying, and they seemed that they want to play again, but the shaking hands part was very hard.

Seniors had mixed reactions, but mainly positive. They said connecting the dots was fun, and it was nice that I could see the kids doing it first and then learn how to do it myself. So, that's where I thought that a collaborative tutorial could have been very helpful. Also another one said controllers were hard, and also some people enjoyed the Swiss view.

Lastly, let me share with your findings from three different points of views. One from user testing, and from intergenerational experience, and also from VR accessibility. So, from the standpoint of user testing, teaching hand controllers are very very hard, so I felt like I should have practiced the script more and created a better template script. Also, as I said before, some people hesitate to do even basic motions like looking around and extending arms, I should have explained that better.

Actually in this test we used a tablet to look into the VR environment without us wearing headset, and that saved us a lot of time to do troubleshooting when they are having some issues. From the standpoint of intergenerational experiences, some in our group said that it is possible that it was easier to talk via VR than in real life. And yeah, controllers are especially hard for seniors, because kids tend to be familiar with the game controllers.

We felt like tutorials and us just explaining were not enough, and it would be better if the tutorial requires coworking, but, once they get used to the controls, they had a great interaction between them. And the accessibility? Yes, tutorials are needed and fewer buttons are better. Because VRTogether software doesn't require too many buttons, in the forest connecting the dots theme, it doesn't require any buttons, and mostly it uses just one button, and that seem to have helped the participant a lot.

I also felt the controller part was hard, and also obviously there are a lot of people who don't use, cannot use hand controllers, so I felt like the VR experiences should be playable without hands. This is what I generally felt, I have been feeling, but some VR experiences tend to be limited according to their users' physical ability, but I feel like VR should instead extend their ability, like, for example, in VR we can fly around, while in physical world we can never do.

And so, instead of only reflecting the.real world ability, I thought it would be fun, and it'll make it more accessible, to explore all sorts of different ways to extend the ability.

This test was mainly to evaluate the feasibility of VRTogether improving the lives of seniors and kids, but, as Equal Entry, we are very interested in the accessibility of VR software itself, and to explore that we have been doing several different projects, let me introduce you to some in the remaining time.

So, we tried a web framework called A-Frame, and it lets you build 3D environment on browser, and we created this conference building. On the slide I have pasted the image of bird's eye view of the building drawing. We have a lounge and a conference room and convenience store. It's sort of like a meetup building, and it was built by our talented developer, Celso.

He has an architecture background and also he's a web developer, so he was perfect From the entrance, users can use the door on the right to go to the convenience store, and the door on the left to go to the conference. We did an onsite test in New York and Tokyo, we recruited some people with vision disabilities who are interested in VR. On the top left we have the image of Kato Sun, who's blind and he's in Tokyo.

We tested with him in Tokyo if our convenience store is usable for him, and we have documented all that in our article, same as the seniors in his article. So, if you're curious, please check it out. And also, I think about a year ago, Thomas mentioned in this A11y VR event, about alternative text for VR 3D objects, because in 2D images, we have alt text, but in 3d what do we have?

Like, for example, maybe many of you are familiar with the old attributes for image on web, and this way blind users can read with screen readers what the image is. But, for 3D object, for example, we have a 3D object of a dog, Shiba dog, on this slide, but how can they read it? Currently a file format called glTF is the standard for 3D objects, like png or jpeg. In the past meetup, I think Thomas mentioned that we can use that label feature on Hubs, but we didn't have a way to embed the alt attributes into the 3D file itself, so we have documented how we do that in the article.

Also, lastly, I want to introduce, as we did the seniors research we learned a lot of accessibility features of VR, and Meta has a set of guidelines for virtual reality software called the Virtual Reality Checks, and it is a guideline for Meta Quest software developers, and there are many checks, but nine of them are related to accessibility. They're just recommendations and not requirements at the moment, but at least they have something, and nine of them are about captions and accessible text and multiple cues, multiple feedback styles, and also a software should be playable with one hand.

Unfortunately, there's no recommendation for no hand controls. Also we have the display setting, and something about colorblindness, and rotating the view without physical movement, and multiple locomotion style, which is teleportation or free locomotion. Also everything should be playable with seated position. We focused on the seventh one, rotation.

Users should be able to move without physical neck head movement. Meta only says this, it should be playable. Application should provide the user with the option to rotate their view without physically moving their head or neck. But we felt like, how do we do it?

They don't document in their guidelines. And this might be straightforward for skilled developers, but considering there'll be more and more new developers, we thought it should be documented on how we can actually implement that. Of course, it varies depending on the development environment, but we tried to document for the major platform, so we chose Unity and created a simple step by step implementation instruction. We feel like simple and clear instructions are important for accessible VR to be a standard thing.

To start simple, we focused on one of the possible solutions called snap turning. there are other possible solutions like gaze control or voice control, like saying, Look to the right, Look to the left, and rotating the view without turning your head. Also we have eye tracking in the latest devices, but our developer, called Ahsan, he helped us document how to implement the snap turning. In our article we explained what it is, and in a video and text, and in a screenshot of Unity editor, we explained step by step on how to actually use your thumb stick to rotate your view without actually rotating your head.

We have published the article to share with you guys. If you are building Unity development, you can use our resources to conform to the seventh check of Meta's accessibility guidelines. And, we're trying to do more other stuff. That was a little bit of our other activities.

That was it, so thank you for joining. Also, I would like to thank Meryl and everyone involved for organizing this Meetup, and extra thanks to Meryl for even helping me fix my English and improve the slide deck in general. Awesome. Thank you.

All right. Thank you so much, Kenji, and let's also give Kenji extra praise, this is his first presentation in English to a group, so it came across really great. Thank you so much, Kenji. I wanted to just share my own personal experience of doing the research study with the senior citizens and the children.

If it's not my most memorable event, it's has got to be one of my most memorable events from the last year. It was so interesting to work with both of these populations that, a lot of times in the years I've worked in technology, we don't necessarily focus on either of those user experiences. So, it was very interesting to get to work on this project as well and learned a lot. I wanna also say that, Azalee and VRTogether, we really appreciate all the work we got to do with them and to actually make the project happen.

I think it's pretty unique to be able to get to bring seniors and children together to do this type of experimental research, and the design of VRTogether really worked quite well, because we went into the research study not really knowing, because we hadn't done this type of research study before, how it would work. I think one great thing to look at in the simplicity of design is the way that the experiences are able to be accessed in a more simple manner made it more fun for the participants, and we've done a different study with a very involved controller experience, and, especially for people who are older, it's just not intuitive at all, and so the results were frustration or confusion.

So, it was a really great experience. I see that we've had some great questions for Kenji, and as we mentioned, we also have Josh Grisdale here from Azalee, and Richard Hoagland from VRTogether. The first one we had was, Would it be easier for some to get the concept of controlling a VR avatar if they could see their character from an outside view, like controlling a puppet instead of being first person? I see, so instead of first person, you mean that third person point of view?

We can't say anything for something we haven't tried, but it's very possible, because it's possible that one of the reasons why they got confused could be because they were in person view, and they could only see their hands. But we definitely need to try, if that helps them, in the next test. And maybe Richard has something, in his experience or opinion, about the point of view. Hi everyone.

Yes, I'm Richard Hoagland from VRTogether, and as everyone else said, Kenji, that was a great talk. Thank you. It's an interesting question. The two challenges that we've seen, one of which I think it addresses, and Kenji spoke to, is the understanding of like their own body as a virtual body in space.

So, if people seem to struggle with the kind of transition from, I'm in the real world as normal, or as it is, and then I go to this virtual space and the body doesn't come with me, it doesn't translate with me, so people have confusion, like whether the hands in front of them are in fact their hands, or that there's some other hands there. So, I think it would be interesting to see if that would help with understanding, kind of establishing that connection to the virtual avatar that they're embodying, or something even like a mirror that I know that is now in the Meta home space, where they could just see the one-to-one interaction.

The other challenge that I think you all touched on, and we've seen is just the trouble around the controllers and their, I guess, physical self, so I would love to know techniques for that, like the example you provided, which was so interesting about putting the controller down to shake the hand. That's certainly another challenge there. Let's see. Have you considered reevaluating the experience where hand controllers are not required?

For example, there's a representation of their hands in arms in VR? Yeah. I've been recently testing the new Oculus feature of hand track, and I am personally definitely really curious about how they would perform in that feature. But Richard, do you think you can implement the hand tracking feature, if that can be tested?

I'm very very curious about reevaluating the test, of course, yeah, because controller was very hard.

Were you curious about whether we would...? Yeah. Hand tracking, is that feasible? I think they have provided a lot of tools.

I will admit that I have not, even in a prototype, played around with it. I agree, it can't be so much more intuitive. One of the reasons we chose controllers initially is for the haptic feedback, because we're interested in holding hands, but it's like, is that worth it with all of the accessibility challenges? Is putting all that effort into having haptics for one thing worth the challenges that people face to use it.

This is why I really like something like this research, because I think it helps guide that question, and I'm thinking, maybe not! The haptics at this point, or it could be an additional feature as opposed to something like at the foundational level, this is what we focus on. I'll just throw in there for the hand tracking, for myself, and technology keeps advancing, but when I try to do some of the demos, like hand lab interactions on Oculus, or some of the things available, I never had the dexterity.

It might be my environment or like low lighting, but I never really felt like the fidelity was good enough for me. I would get more frustrated with, it's not working seamlessly. So, like, I prefer the hand controller for how it is now, but that's something that I guess, can be changing all the time. I'm going to call on Dylan Fox now from XR Access, and we were mentioning XR Access multiple times here, so thanks.

Hi Dylan. Hey Thomas, hey Kenju, great job on the presentation here. Something I'm curious about is, you mentioned this idea of kind of a collaborative tutorial of, we're talking about people's first time in VR, having them do it with other people as opposed to like getting them set up on their own, and used to all the buttons and everything before you then put them into a shared environment, and I guess I'd love to hear some more of your thinking on the pros and cons of that, because I could see on the one hand, it could be hard, there's more stimulation, there's more people to manage at the same time, it's hard to give people individual attention.

But, on the flip side, you had trouble with getting people to like, look around, extend their arms, do things. I think if somebody could be there with them in VR, maybe that's something that could get them to much more naturally do that type of interaction. So, I was just curious how you saw that playing out and, what you would think about in terms of the ideal version of it for the future.

Yeah, I totally agree on these pros and cons of our collaborative tutorials, and that was just one idea I came up in my mind while I was doing the test. Obviously, we need to test to see how they would react. In the testing we did, I really felt like it was a shame that there was literally no conversation between them while kids were getting good at it, and seniors were not, and I felt like ideal instruction could be, tutorial could be very very basic, very very step by step, visual and text and audio instruction, because I have seen many different tutorials for VR and games, but it still assumes a certain amount of knowledge of VR or games, because normally games are created for people who like games.

But I saw one VR experience that has a really really basic tutorial, with a real image of hand control, and hand floating in the air, and explaining this is how you hold controllers and this is how you press the control, so I'm sure that collaborative tutorial could create confusion, but I guess it depends on how well you make the tutorial. So, we need to test. And, not very good, precise answer. And we have another question, was there any assessment made of the user's functional vision, as this would also significantly impact on their experience?

So, like how well their vision was? Of the participants? Yeah, actually we did not ask that. I guess we need to ask that to Josh if they knew about their participant's vision.

Hello everyone. Thanks for the presentation, Kenji. It's an interesting conversation. First of all, I should explain that I work in the marketing department for our company, I'm not really hands on with the actual clients, and so I wasn't involved in the selection of the people who participated.

One person did have more limited movement than the others. One, I think, had a stroke previously. But there was no mention of visual abilities of the participants, although definitely something that I think we should bring up in the future. Going back to the last question, if it's okay, I think that probably one of the things that needs to be done for the onboarding, for the seniors in particular, is maybe even a basic, what is VR?

I think a lot of them probably have absolutely no idea what VR is in general, so adding that to it. But, in the end, I think it was very very positive. We were a little bit worried actually about how the reactions were going to be for the seniors, particularly if there was going to be any disorientation. We were careful not to choose anybody with dementia in case it was a trigger, or something like that, but the overall impression was that we really really had a good time.

We just wished we could practice more. So, they were very positive in that they wanted to not just... It was too bad that they couldn't do more. It was more like I wanted to practice more so I can have more fun next time, so it was a really positive if that answers both those things in one step.

Yeah, that's great. I think that's the note on the onboarding and the sort of getting familiar is something that, due to the time that we did, we just sort of dove in. And so, again, that's where I was saying, this is good, the way that VRTogether is designed, because we still had success. I do think a lot of application experiences would just totally fail, and that because they are so complex.

Still though, like something to take in for thinking about doing a user research study as well, is that we really didn't have any... We did have a little write up, but I think Kenji had that in the presentation, that we should improve the template for when you do a research study, make sure you really familiarize and orientate to the device, because, yeah, it's very unknown. We're going to take one last question, because we are getting close to time, and I like this question.

Could there be features that kids could unlock by earning points for interacting or guiding or teaching others?

I guess it's a question for the software itself, and Richard was raising hands too. Yeah, I'm happy to answer, and I think I could discuss what I raised my hand about as well. So, there's a researcher that I met recently, Jose Rodriguez, and their PhD was around a series of.... I'm going to do a terrible job of describing this, but they had young kids and older adults learning something, that neither was familiar with, at the same time.

And it wasn't virtual reality, but talking to them was really interesting about that project. And so, hearing what you all are talking about today, I just wanted to share that because I thought it was really interesting, that idea of like, if two people aren't terribly familiar with something, and they're at kind of the same level, and you can have them collaboratively learn and explore that. I guess that leads into the question about points.

We haven't considered it, but I would love the idea, because my background is in gaming and I'm a big believer and fan of gaming or gamification, when it's done well, as a way of motivating people or inspiring them to do things that, if it's beneficial to the experience or to others to provide that motivation for them or the positive reinforcement, Oh, I got this thing that I enjoy, and I enjoy doing this, so I'll do it more.

All of this, to me just I guess conceptually, it's very fascinating, the idea of, say, incentivizing the younger group through awards or points to help in the process of teaching that, and then also using the experience not just to provide an experience, but like the experience of learning how to do this thing together as, what I think are the best tutorials and interactive things, which is when you don't realize it's a tutorial, you end up learning how to do the thing, and you didn't even know that the whole first part of it was onboarding you into it.

So, yeah, I guess the short answer, I love the idea. We discussed it at this point, but certainly are interested in it. Actually, before the testing, I was very very worried that kids might get bored easily because we don't have any points system or any, like, certain achievements, so my worry was that. Seniors enjoyed the simple environment, but I was worried about the kids.

But, it turned out that kids really enjoyed without any purpose or points, maybe because it was their first experience, but the kids seemed to enjoy like simple thing about being on the beach and throwing the blocks for no reason. So, yeah, that was a good surprise, so, at a certain point, kids can enjoy without certain goals or points, but yet maybe it's different if they do it again and again, so that'll be a very interesting thing to explore.

I think.