The Latest XR Accessibility Findings | Accessibility NYC Meetup Recap

THOMAS: And with that, I think I'll just get started. This disclaimer here -- this was an event that happened last week. And this is sort of my... Best idea of a summary of some of my key takeaways.

What's great about this W3C event is all of the full talks are linked to from the W3C site. So I'm just taking small excerpts from each of the presentations, but you can easily get access to all of the materials. So the event itself was called the Inclusive Design for Immersive Web Standards. It was hosted in Seattle, Washington.

And it was Tuesday and Wednesday of last week, at a place called Pluto VR. So Pluto VR -- they are doing telepresence VR, and they're very interested in being a leader in accessibility. So it was great that they were able to offer that space. And bring all these people from around the world together.

From the New York City contingent -- and I think several people here are in the audience here -- these were -- at least the ones I was aware of. Andres Cuervo, from Movable Ink, Markku Hakkinen, Regine Gilbert, Roland Dubois, Suzanne Taylor, and Xian Horn from Changeblazer. So we had a great representation from New York and all the people who are here tonight would be great to talk to after the event for their impressions as well.

So what did we kick off with? Leonie Watson, who, if you've come to our meetup, Leonie has actually presented here at accessibility New York City. And she was also the chairperson for the event. So she started the event with why we're making the immersive web accessible to all.

And it really starts with this opportunity with the web being a standard that's open, and many people can build on top of. It's kind of the most exciting area -- or a lot of us that were participating in this group think it's the most exciting area, because the history of the web has had great accessibility. It's not locked in. So there's a lot of opportunities, and there's a lot of work ahead of us.

The next piece that was quite interesting was Matt May from Adobe. He talked about some of the key principles of making web experiences accessible. I was particularly interested in just considering the history of the W3C, thinking about how we did get to the creation of standards, how now we can build accessible websites, use things like ARIA. It really started back in 1990, with Tim Berners-Lee, inventing HTML, HTTP.

1994 is when the W3C was founded. And 1997 is actually when the accessibility interest group -- so most of the work now that many of us will be familiar with -- started from that group in 1997. Very cool.

And we had dinner with the lady who was once a co-writer!

THOMAS: Yes. Wendy Chisholm was one of the people who was originally there in the W3C, working from the Trace Center. I didn't know that WCAG or Web Content Accessibility Guidelines -- a lot of that had come from the Trace Center at the University of Austin. We next had Kip Gilbert from Mozilla.

Lots of companies represented. All of the browser vendors had someone there except for Apple. But we had... His discussion was really: What's the architecture, currently?

What's available, and where would accessibility fit into the APIs that are being developed currently at the W3C? I think one of his key points or one of my takeaways is that WebVR, which is what most of these web XR experiences have been built on, is going away, and there's a new standard, WebXR. So they've decided to take the AR, augmented reality, VR, virtual reality, and combine it into one new spec that's coming out to all the browsers.

Very technical. Most of my stuff is gonna be demos, but one of the technical points that's interesting in the changes here is there is gonna be a way to have the browser know what a "primary action" is that can be performed. This is similar to a lot of assistive technologies. You need to be able to click a button on behalf of someone or cause an action to occur.

So there is this primary action definition in the new device API. There's a laser pointer that works with gaze-based. Can be done with the track controller. Can be done with the 2D computer screen.

And then there's this XR input source event that the browser is involved in, but he pointed out that they're currently not available to detect what gets hit by the rays. So detecting objects in that immersive world -- not part of the spec yet. Then we had Josh O'Connor from the W3C. So he was talking about XR accessibility user needs.

He's put together with people in his working group a pretty comprehensive starting point for the considerations for the use cases, and he's very open to getting feedback and collaboration. At the end of the presentation, there'll be lots of opportunities of how you can get involved, if you have an interest in this area. One of the areas that popped out for me, because I'm someone who in my day-to-day work -- I'm trying to work with companies to make changes in their technology, and one of the ways we do that is through checkpoints or through requirements.

So inside of that longer document around the user experience for XR, they have -- I believe it's 35 checkpoints currently defined, that are heavily in draft, but interesting to look at and consider. So there's things about making sure that... Let's try... The XR content is compatible with a range of AT.

Avoiding interactions that trigger simulation sickness. There was a lot of discussion around vestibular disorders and the idea that in XR that's gonna be something we need to think about even more. But there's a whole bunch of them. And I've linked to that full document.

And you can dig into that. Here's me. I did some discussions on just existing prototypes. We're actually -- my company, Equal Entry -- gonna be doing a presentation at CSUN next year, the big accessibility conference in California, around the accessibility of immersive media, like 360 VR.

So one of the things that I worked on with James Herndon on my team was looking at the Anne Frank House. It's a current virtual reality experience. I've got audio description here I'm gonna read. So we're in an attic with floral wallpaper and a gray tinted window.

To our right, a wooden banister blocks the stairs leading up to the room. To our left a table is covered with manuscripts and a non-electronic bull horn hangs from a wall. A single light hangs from a ceiling. Straight ahead, a bookshelf is half full with bound materials.

On screen text reads "point to click and move". That was just a simulation of this video playing, but we were working with a person who was blind, wearing the headset. And just considering the idea of the same way that you might be using a technology out in the real world to get sighted assistance to describe what's being seen, you could do the same thing in a virtual world. So currently the Diary of Anne Frank -- or sorry, the Anne Frank House experience doesn't have any VR description.

There's no interactivity for a person who's blind to know where to click on the screen. None of that is currently supported. But we want to just explore the idea of what could you do with on-demand audio description. And I actually have my sister here tonight.

It's her first time coming to accessibility New York. She's part of this scenario here. This is an app called Wander. And I thought it's a good UI pattern to look at for thinking about immersive media descriptions.

So in this screenshot, we basically have my father here, represented by this avatar, and my sister here represented by this avatar. And you have basically little circle pointers that show where each person is looking. And so the colors are associated with the people that are in the environment. So my dad basically took us to York, South Carolina, which neither of us had ever been to, and showed us our grandfather's home.

In virtual reality. And could kind of lead us around and describe the walkways and the paths. And it was a very compelling experience. Our dad actually bought an Oculus after using it.

So I told the people from Oculus it's definitely a great demo. Because he wanted it immediately after that. But I'm gonna show you the video here. This is just my father and I walking down the street inside of York, South Carolina.

So there's a VR pointer aimed at a white store front labeled Sylvia and the VR pointer then moves into the street. It highlights and selects an arrow that moves us farther up and down the street. So basically inside of this collaborative experience, I can actually see where my father's pointing or sister is pointing. And then you can navigate and get to that area.

So this is something that I thought was an interesting idea. If it was applied to working potentially with someone low vision or blind, inside of any VR experience, actually being able to see what the person is seeing, where they're looking, and being able to provide assistance or descriptions. So we're gonna keep working on that, and next year we'll have a full presentation. We're doing user studies with a set of different users to try out different techniques.

Okay. I think this is the next one. Yeah. The next one was Zohar Gan.

He came from Israel. So we had people really from all over the world at this conference. He has a company called Accessible Realities. And one of the areas he was working on was on the Unreal Engine, making a way to provide semantics or accessibility semantics through that model.

I just picked one demo that I thought was pretty cool, showing more of an AR scenario. But he has identification of different objects in the room. And I'm gonna just show this video. So after a scan occurs with his technology, indicated with a side effect, on screen text labels show three chairs, a dining table, two cups, and a drinking glass.

We zoom closer to the table and scan again. This time, onscreen text labels three chairs, three cups, a dining table, and a cell phone. That'll be happening in a moment. Right.

So there's just a new object recognition from a new vantage point. And then next the filmmaker accesses the cell phone, flips it over to reveal an image of a white dog, and then begins the next scan. Once complete, onscreen text labels it a domestic dog. Throughout the video, an onscreen watermark reads "phone video recording at 2x speed".

So this is at double speed, but it illustrates the idea of, I guess, using these AI technologies. If you've seen SeeingAI from Microsoft, it has some functionality kind of like this for exploring pictures, but this idea was really like being able to get that information repeatedly and navigate in the world and ideally get more and more real scanning. Zohar Gan has a spec and has put in a lot of thought around a data model for semantics XR.

So that's something you can look into a lot more in detail and he has full details for what he thinks about that data model. Okay. The next one was Chris Joel, from Google. And he was talking about the model viewer that's part of HTML5.

So he was explaining that one of the core use cases currently on the web for basically this XR content would be things like IKEA and Pottery Barn and different furniture stores, where these ideas of commerce... People want to explore objects, rotate them, so model viewer... Current use case is mostly around allowing companies like that to get these 3D models to be easily uploaded on the web. He had this, I think, a great quote.

And this is a good one to think about. That this new term or word -- to become familiar with. GLTF. It's a file format that's part of...

Basically he's comparing it to mp3 is to audio. h.264 is to video and jpeg is to images, glTF is to 3D. So basically that astronaut object is a glTF file that gets referenced. And he was showing very simple code, but I was excited to see...

Because I've seen so little progress currently in what can be done accessibly... There is an alt attribute. So our old friend the alt attribute... On this model viewer component.

So in the current released model, I tried setting this. You can load it up with VoiceOver, a screen reader, and you will get that description. They also are doing some work to tell you when you've rotated the model to the back side or to some different angles. And they had some discussion in the spec of how they would add additional labels to more detailed parts of the object.

So potentially you might want to label the helmet separate from the gloves, separate from the boots. So they have that in a spec phase, but not in an implemented phase. But it was very interesting. Okay.

And then the next presentation... I really loved this tool. So this was authoring immersive environments. Mozilla has this product called Hub and a product called Spoke.

And Liv Erickson from Mozilla talked about some of their plans for what they're doing to make that environment accessible. It's probably one of the best demo apps for WebVR. Because it does use so much of the functionality available in the spec. So some of the things she said that Mozilla is looking ahead to focus on inside of this world is to make sure this idea of spatial accessibility structures -- the fact that in the 3D world, we have to have relationships of what's in front of us, behind us, they want to be able to set alternate text onto different objects you put into the world.

They want to have application-specific ways to give information. Especially captioning. So when we're talking, it's a social world where you can meet up with people and talk. So the idea that -- like we have closed captioning at this event.

We would be able to have captioning inside of the immersive world. And they're very open to continue learning and adapting to new information. So I made a couple videos, since seeing the presentation, just to illustrate a few of the points about using Mozilla Hubs for social VR, and just to help further the discussion of what could be possible. So I made a room -- Accessibility New York City, Cooper Hewitt.

Very on brand here. But this is showing... This is functionality inside of Hubs, where you can import objects. So I'm looking in the art tab.

And I'm putting into my virtual room a painting or a wall painting. So notice when I created it on the back side of it there's no image. So I have to flip it around. And then I have basically like...

Sort of a digitized rendering of the Mona Lisa that I can pull in. Now, for someone who is blind, that might come into this room, understanding what that object is -- right? We could either have someone else in the room, walking around, describing what that object is, or we could have something like an alt text for the GLTF format, to give it a label, allow someone to ask what that object is, and get that information back.

Just wanted to show -- we have the same problem like we do on the web. Where you could have the same objects having the same label. So, you know, one idea that I had, that I thought... Well, we should try to reuse all the different labels that already exist in the store...

Because those are semantic discoverable items... But here we can see that in this particular instance, both of those objects are named "wall painting". So if we were gonna use just the information that came from the store, there's really no additional details about, like, well, what is that a painting of? And as we know with alt text, the context is always really important.

So depending on the room that we're creating, whether or not we want to describe that one wall painting has mountains and a sun on it, and one looks like the Mona Lisa, that's additional functionality or additional metadata that needs to be discovered and discussed. How could that be made accessible and usable by people that can't see these objects? And then this is me placing that on the wall next to the Mona Lisa.

It's kind of a fragile wall painting. But yeah. So you basically can set up your room, and I'll put a link to this room. You can go on your computer and load up the room.

You don't have to be inside of virtual reality to do these experiences, so that's another great thing about Hubs. You can do it on the desktop computer. Walk around in the 2D world and still see all the objects and control the functionality. The next demo...

Really cool technology from the Microsoft research group, SeeingVR, is a set of tools. This is basically based off of Unity, but can apply to WebXR as well. And these tools are really focused on low vision accessibility use cases. So I think it's at least a suite of nine tools.

I think it's even more. But I picked just two of the tools, to show you a little demo of their ideas about the functionality. So this is gonna be a bifocal... Oops.

No, sorry. This is a depth measurement tool. So this is... Many people with low vision have difficulty with depth perception and actually grabbing virtual objects.

So their add-on actually adds a pointer or selection UI that makes it easier to grab objects. So you can hit test and pick up objects, potentially with a high contrast wand, or something easy to see than maybe the hands or what's been generated inside of the virtual environment. So if you look at hubs, for example, and manipulating those paintings, they can be thin and difficult to reach. So this can be a tool where maybe we can have an add- on for people who need it, so we don't need to have such fine dexterity to grab objects.

We can define this tool that lets us select an object to manipulate it. I'll show this video from down here. All right. Here's the bifocal example.

So this was the idea of having bifocals, just like... If you're wearing them, you can have part of the world not magnified, and then have a section of the viewport magnified. So they're showing picking up a book and maybe looking at the book through the bifocal part to magnify that area. And they're showing you can adjust...

Because it's in VR. You can move around where that magnifier is. And so this is a tool that you could add into a Unity project and get this functionality, and potentially allow someone with low vision to better interact inside of that world. One of the other ideas that they had, that I thought was very cool, and I tried it out, just myself, with SeeingAI, the Microsoft app...

So this is taking a picture of my room in Mozilla Hubs, with the SeeingAI app on my iPhone. And so with SeeingAI processing the virtual objects, it says it seems to be a flower pot, house plant, design plant, interior. So this is sort of the current state of SeeingAI. It can kind of sometimes guess, you know, rough pictures.

Not a very verbose or maybe meaningful description, but it's exciting to see that that's progressing. And that's part of the research that they were discussing, as well as the ability to improve the AI algorithms to automatically detect virtual objects and label them for people. And also the ability to send pictures from inside virtual reality, to a remote service, to be described. And then have that description come back to the person, while they stay inside the virtual environment.

So they were doing the AI version and the human-assisted version as demos as well. Next we had a spatialized audio on the web. So Chris Wilson from Google talked about that. He had an example of showing the complexities of what can be done with the WebAudio APIs now.

He built a vocoder, using just the WebAudio technologies, and he said it's currently used as a stress test by a lot of the browsers, to see how well they perform. Because it's doing a lot of visualizations and sound manipulation. So one of the things he talked about was... There was a lot of stuff going on in audio.

But one of the areas that's kind of current and advanced for improving the sound sphere is this idea of Ambisonics. So it's full sphere surround sound format that you can use to encode and decode 3D audio streams, and he talked about... There's a Chrome plugin called Omnitone that you can use to actually listen to some demos now. Get an idea of maybe how more immersive that can sound.

I made another quick short video of, like, what I currently see in immersive XR. A lot of times, all that's being used spatially to tell where people are is if there's someone to the right of you or to the left of you. If you go to Hubs and there's another person in the room, they can hear your voice, and depending on where you're standing and moving in the space, they can hear to the right or to the left.

So in this case, working with James, I'm saying: I'm going to walk to one side of you. Start talking. Turn to the side where you hear the audio. And let me know if you can hear me from where I'm talking from.

So then James turns around and he can hear the audio changing. To come not out of just one side of the ear, but to come kind of equally into both ears. To know that he's facing me. And we had had some success with that, also testing with a member of our team, Sofia.

And altspace VR, which is a different social VR world -- and it's kind of interesting to say that just with spatial audio, you could have someone follow you around in this virtual space. You know, stay with your avatar. They don't necessarily need to see what's happening on the screen, if they can hear you calling from an area. They can use that as a location finder.

So... Cool. And there was some discussion around... We get excited about spatialized audio.

There's all these new things we can build. Let's think about, like, one, people who are deaf, or also think about people who can only hear out of one ear. So there's always, for accessibility considerations we need to make, we can't have any of these technologies be the sole way of accessing information. So that, I thought, was an interesting one about...

Okay. How can we consider directing people's attention if they can't hear the spatialized audio cue? Which tees up this presentation from Wendy Dannels, who is a professor from the Rochester Institute of Technology. She's doing research with the HoloLens, with students who are going into a planetarium, and she's exploring different ways to provide -- using XR -- captions and Sign Language for the students, so that they can watch the planetarium presentation, but also get the captions or get the communication.

So she talked about... And I would imagine this would apply here to the Cooper Hewitt. Some of the current methods that exist for providing access to, like, an installation, would be... You know, one, a person comes to the event.

There's no assistive technologies. There's no accommodation. So when the person is explaining something about the picture, they just don't... They're confused.

They don't have a facilitation. The second one is showing that oftentimes the interpreter might be standing off maybe to the left of the painting, and the person describing it would be on the right. So you have to direct your gaze from looking at the object being described, maybe looking at the person talking about it, and then looking to your interpreter. You could have someone like Mirabai Knight, on site, realtime human captionist.

So you might be getting a text stream of what the person is saying. Presenting about the art piece. But you might have to look at your cell phone, or like a computer, to read the text. So you're basically changing your gaze...

Changing your focus and attention. And then she gives a few other examples of just... You know, it can get live interpreters and a captionist, and she was explaining that if we're just building these things without XR, it can be very difficult to get everything positioned in a place where you can get access to all of that information at once, without looking around lots of places. So her technology -- and their test -- was just to look at how they could allow someone to position a virtual interpreter inside of the device.

So you could actually choose where the Sign Language interpreter gets placed. It could be to the right of the artwork, to the left. It could be all depending on that current moment, or that current experience. They were talking about, in the dark -- so the planetarium is often in the dark.

So you would already be losing the sight lines to a Sign Language interpreter. So that's a benefit of XR technologies, that you would be able to see the interpreter through, like, a lens display, and still see the rest of the environment in a dark display. And it could be that the artwork is just very far away from where you are. So coming from that, I have a couple demonstration videos that help illustrate what a few of these technologies could look like.

So Melina Mohme from IRT had a talk about subtitles for 360 degree media. And I will show you a couple examples. It's in German. So we might need Roland's help.

But the idea is that when we load up this video, the first video is showing that there's a directional arrow pointing to the right. So there's a caption of what the person is saying. But the arrow is saying that the person speaking -- or where the sound is coming from -- is not in your field of vision. So when we see the right arrow...

Oops. When we see the right arrow, it's our cue to turn until we see the arrow disappear. When the arrow disappears, it means the person that the captions are being displayed for is in our field of view and speaking. We turn our head to the right.

Now we get an arrow pointing us to the left, to say you should direct your attention back to the left if you want to see the person that's speaking. So I liked that design. You know, it makes sense. And you could see potentially that same design working in, like, the Hub's environment as well.

So for someone that couldn't hear the spatial audio cues, you could have the directional arrows to tell you where the sound is coming from. This was me checking... I tried the HoloLens while I was in Seattle. And this is...

The idea of: How do you position the ASL interpreter? So some of the research that Wendy was talking about as well -- this shows one illustration of how you could basically have settings, configuration options, for your interpreter. Choose the size of the interpreter window. And then you actually can position the interpreter window as well.

They did say they usually keep it locked in a specific part of the field of view, and it's also using the arrow pointers, again, to tell you that the interpretation that's being done is potentially for the sound in that direction. So it's using still the arrows to direct your attention, but it's overlaying an ASL interpreter for people that would prefer that type of communication, versus text communication. So I thought that was very cool.

All right. And now we have Roland DuBois, who is here. We've worked together on multiple XR accessibility projects. He's been in this space for many years.

Presented at one of the previous W3C meetings that they had about XR access. I was very interested in his talk. He focused one of the talks on making motricity accessible in XR. And this is around the idea of one of the challenges that he was considering...

Was this idea that in a lot of XR devices, there's an assumption that people can move their head. And that's not an assumption that we can make about every user. So if the lowest form input that's required in XR is that you can move your head to the left and the right, potentially look behind you, that's gonna leave some people out. So he built a working demo, won an MIT hackathon and awards for this, but he worked out the idea of the sip and puff -- if you're familiar with the sip and puff interface, it's also a switch interface, but you have the ability to just send some inputs of on and off.

And he used a short and long and continuous breaths as signals that can be controlled with the gamepad API from the W3C standards. So this is a lot of different videos of pieces that he worked on. I guess I like to focus in on this video up here at the top, showing... There's a user in a wheelchair.

They're not moving their head. But they have the ability with this selection UI to actually have the selection and the view tilt, based off of those controls. And those can be controlled with the sip and puff device, to actually allow someone, without moving their head, to rotate the views, to actually navigate in the views. So I've got a lot more information on it, but super cool.

And obviously a user that we need to be considering as well, as we go into this world of XR. All right. We're nearing the end here. There was a panel discussion about assistive technologies for XR.

And I just picked up the part that I really enjoyed from Mark Hakkonen. So he was talking about -- one of the discussions in the W3C is: Let's not repeat ARIA for inclusive XR. So he was saying... You know, ARIA 1.1 is like accessibility code bandages or Band-Aids.

It's like... Flexibility and comfort your users will love, with added properties for serious accessibility. 81 semantic roles. See back for details.

So he's kind of pointing out... When we even look at describing a user interface, right now there's 81 sort of semantic terms. And trying to teach developers or all get a shared understanding of what those 81 roles are and how they should be consistently applied is difficult. And you multiply that into XR, what's a role?

Do you have a role for a desk? Avatar? A person? A chair?

There's so many different things. So part of the discussion was just really the need to rethink assistive technologies and think about different ways to do that. And maybe take some lessons learned from ARIA, as we go along this model. We ended with some road mapping ideas from the workshop.

So Dominique Hazel-Massieux. One of the organizers from the W3C. He did a great job. Basically if you go to the website and want to dig into any of the different talks from the workshop, he put all of that together.

Got it all organized. He collected some of our thoughts for what needs to happen next. So some of the ideas discussed for future work with the W3C was: Making accessible GLTF files, those files that we said are like the JPEG of images. What other metadata can we add in to those?

Accessible HTML support for GLTF. Best practices for making web components or GUI components in WebXR accessible. Considering the complete content chain. Semantic XR data model.

Sensory and experiential annotations. Annotating annotations, so the idea that if we have AI creating the baseline of annotations, how could we override or have humans that have a better understanding of what it is include those annotations for other people? And a framework for reporting accessibility issues. One of the things that comes out of this -- there's a lot of different manufacturers making these devices.

They're all using different technologies. They're changing every year. How do we make sure that those companies do build in accessibility? But then also consider standards like...

At what point can we have standards across these devices? And at what point can we start expecting when these devices come out -- they do work for people with disabilities? And then my last example, before we do the Q and A, was I just did another example inside of the Hubs. And I think shows a challenge you have currently with WebXR.

I can choose images from a Google Image Search, and insert them into my Hubs. So this is just taking a regular image that would have an alt text, if we were looking at this image on the web. But it's not gonna have that information when we bring it from a website where we can set an alt into this virtual environment. So this was actually an example of the exhibit.

The access+ability exhibit that was here at Cooper Hewitt. And I was able to sync up with Ruth to get a link to the particular image that I was showing in that demo. So this was the suit prototype. And the Cooper- Hewitt has a great data model where you can get information on every object that's in the collection.

So I think it's a really cool idea to see -- this is all web technologies. In theory, you could have these renderings or these images -- if we had the plumbing for accessibility, we would be able to connect, in that virtual environment, pull in all this information, get all these rich details about the object. And it's very potentially close here at the Cooper-Hewitt, because they do have this type of model, where all the objects are cataloged and metadata associated.

So it may be a future project. You can get involved. So this is the link to the W3C workshop. All the different decks, all the contact details.

All the participants are also listed in the site. There's at least 50 or 60 people in the site that are interested in this space. There's the immersive web working group, which is from the W3C. And there's also the XR access initiative.

Which is a working group also focused on making augmented and virtual reality accessible to people with all abilities. And so Shiri Azenkot from Cornell Tech, who presented at previous meetups, she's sort of the New York City representative for this group. So if you have interest in any of these areas, it's an open working group. They have a call for participation right now.

And with that, let's go... Thanks for listening! And we'll have some Q and A. (applause)

Questions?

THOMAS: Comments?

JOLY: I had a nitpick. You talked about... We have closed captions in this meetup. Whereas I would say they're open captions.

THOMAS: Open captions. Xian?

Hello! First of all, I want to say thank you. I actually learned more from your presentation as a layman than I did probably in the two days -- because I needed a translator. So you just translated a bunch of things.

Thank you. But also, I wanted to say I think what was really powerful that I took away from it is there are certain things that we don't think about, 'til we're in that scenario. So, for example, the example you gave about... Maybe people can't move their head, or Wendy stuck out to me.

And your presentation stuck out to me, just because you gave so many immersive examples. Sorry, Chuck. You didn't make the cut tonight. But you were there in Seattle.

And no -- I think you really gave us an immersive experience, but also I think Wendy's point, which again... I'm always coming at it as an end user, so forgive me for not being as technical as most of the people in this room. But I thought it was really powerful that she made the point... You know, it's very distracting.

If you're trying to enjoy a piece of art, or a wonderful design, you're not necessarily going to be able to look at the interpreter, look at the captioning, look at everything all at once, and enjoy the artwork in front of you. And so that really stuck with me. And I think it's wonderful that everyone you mentioned has come at it from so many different standpoints, whether it's... Am I saying it...

The ambisonic... You know. Anyway, just to be among people who are so creative and thinking in these ways was very powerful. But I just think that it was really interesting to think about -- even as a person with a disability -- we don't know every disability there is.

So I also feel like I have more of a sense of what some of the limitations are, but then also I got a really powerful sense of what is being done about it. So thank you guys for doing that for us.

THOMAS: Yeah. I was gonna comment too, just mentioning Wendy's work at RIT. So she was working with students. I think...

I'm not sure the grade level. But she pointed out just from her user testing that you need to have a way to lock down these XR devices more. To keep it in sort of a kiosk mode. Because she was saying, just from her user studies, the students were just excited to be playing around with the technology and not staying focused on what she designed.

So I thought that was an interesting observation.

Another add-on to Wendy's talk was very interesting. She was asking questions in the beginning: How we actually enjoy movie theaters, or open plays. And if you have a live interpreter, or any kind of person that requires certain light, you are actually disruptive towards the scene setting. So you have another spot of light.

And that she mentioned that it was majorly embarrassing for her to be in need of something that's distracting other people. And with the XR headset, that kind of fell away. And it kind of empowered her more, being more inclusive and less of a factor that causes more attention than she wants, and is required to have. So it was really interesting to see that kind of aspect to it.

And we don't get that kind of insight unless someone from that side speaks about the friction and the kind of remedies that are currently in place, which, like the XR technology -- will directly take away and make perfect. So it was really, really interesting.

To your point too, I thought it was interesting -- she brought up kids. You know? So you mentioned students. But I remember she specifically said that kids get distracted.

And you give them something as cool as a headset, and I think... Who made the point also that you may want to use this headset. It doesn't mean you want to walk outside with the headset on. Right?

I remember from Connectability Challenge, somebody created something that looked like a Robo... I don't know. Just like a helmet or something. And Gus, who worked with me, who was blind, was like...

I'm not wearing that outside! Like... No way! So I think...

Yeah. It was great to see sort of... Obviously you guys often have to work in a vacuum, to create this, but it was awesome to see kind of the things we could learn about living with the technology or actually using the technology, the things that come up, that we don't necessarily expect. Anyway, thank you.

Hi, Thomas.

THOMAS: Hello.

Thank you very much. This was very informative. I was wondering if you can go into how the device makers themselves for XR experiences have tried to bake in, if they have, kind of standardization of user controls at the device level. Because I know that's a common issue with accessibility that doesn't get talked about enough.

The fact that there are so many different assistive technologies and device types, and each of those come with their own baked-in user controls, and a lot of the times those aren't standardized for accessibility from one to the next. So is that something that's being worked on in the XR community?

THOMAS: Yeah. So I mean, my knowledge is as fresh as whatever release that I've tried of all these different products. So they are obviously constantly updating and innovating. I would say generally most of the work I've seen for accessibility in this space has been app-specific.

And people have coded in support for, like, subtitles. There's not a platform-level way on, like, Oculus, to say: I prefer subtitles. And then as a developer of any application, I could say... Oh, they want subtitles.

Let me turn those on for you. So that kind of idea exists in iOS and Android. But we need that kind of model on these devices. So that then people that are building experiences can implement them in.

But I think definitely those companies are part of the conversation, and are very involved in the W3C group, and in the XR access initiative. So it's encouraging that they have people dedicated on their teams to work on accessibility. But I guess it's just the speed of that. I'm always impatient.

I want it to be accessible now. So a lot of the stuff that I was seeing... I think Hubs is a really cool one to work with, because in WebXR, it does let you do and explore a lot of the different ideas, and it's great that Mozilla has people dedicated to working on the accessibility for that, and has it in their plan. So I think that'll be an interesting place to stay tuned to.

Other questions?

So I think you alluded to this a bit earlier. With... Maybe we could push Oculus to add accessibility at the OS level. What do you think is potentially, like, the highest priority items, in order to push accessibility more?

Kind of like a domino effect, in XR?

THOMAS: I mean, definitely the assumption, I would think, is our legal system here in the United States. Because that's always been something that has helped get accessibility going. I think it's interesting that you have some of these experiences already going into the public school system. So you already have people being excluded immediately now.

So that's kind of my opinion on it. Like, if you're already trying... Or these companies are making money selling to, like, government agencies, that's a good place to potentially really advocate for that. I think the people that are making those procurements need to be educated, that they should demand that.

Get that into their contracts. Request that information. That's definitely one way. But yeah.

As far as the places to start, I was very much... At least starting from what exists on the current web today, iPhone, Android, it's sort of like... Why should we completely start over on these new devices? There's a lot of things that already directly apply, and I think that's something that we'll just have to see as they come out.

But if you guys have any other ideas, I like definitely communicating as much as possible... If you do purchase one of the devices, giving that feedback. It's very important. A lot of people don't realize that there are people working in accessibility in those organizations.

And it really helps to get real feedback and comments from the community. Because if they don't receive those, it's hard for them to make the case internally. So the more that you can communicate those, it helps the people that are working in those organizations.

And just to follow up to that, I'm curious, because you mentioned the US policies that they're behind on. Have you noticed anything -- in your time in Japan, have you noticed them doing anything better that we can learn from, especially in this area?

THOMAS: Still in my learning everything that is happening in Japan... I would say that it's kind of... My experience so far is that Japan looks to America as, like, more of the role model and leader. Because we do have progress being made.

And they don't necessarily have the legal requirements or the societal awareness that technology can have these features built in. So my observation has been... Physical access, the transit, everything about navigating the physical world -- really great and superior. But when you get into the technology parts, they haven't had even some of the work that we've been doing say 10 or 15 years ago.

There's not people scanning the websites or thinking about the screen reader conformance. All right.