Adding Alt Text in Social Virtual Reality

I'm Thomas Logan. I'm the CEO of Equal Entry, [speaking Japanese], accessibility activist, and I'm very passionate about VR, AR, and XR. That's why I've been the organizer of this meetup for a couple years now. And you can find me on LinkedIn or Twitter at @TechThomas.

I was looking back to when this meetup actually started, because time flies. Especially in the COVID times. And we actually started this meetup March 20th of 2020. So we have had this meetup going for a little over two years.

And we've been having events monthly. And I'm really excited to present the information we have for you all today, because I can say from at least where we started two years ago, to where we are now, we're definitely making progress. And for those of you all either on the YouTube stream or here in the Zoom, or watching this later, we want your assistance too, to continue making this a more accessible environment, this idea of using social VR.

So let's just start into what I'm gonna primarily talk about today. And this has been a topic that... When I first started getting interested in VR, as like a starting point of even in my career of working in accessibility, the idea that there's graphical information and people who are blind or low vision don't have access to graphical information most of the time. Visual information.

So... At least on the web, we have this idea of alt text, and probably all of us here on the call have experience working with alt text. It's pretty much one of the oldest principles of accessibility for the web. But how does that work in virtual reality?

And what does virtual reality currently have? Can we actually... Even at the simplest point, at least, label something, or give it a name? And for the most part, unfortunately, no.

So this is an example from what the objects list looks like inside of Mozilla Hubs. So for those of you that have been to a previous event, if you've been to our events, when we host them on Hubs.Mozilla.com, which is the default place that you can host a social VR event, any time an object gets created in the world, and an object can be pretty much literally anything, like a 3D asset, that you pull into virtual reality, in the objects list on Hubs, it gets a name on the screen.

I'm showing reticulum.io...4c43. And reticulum.io...30a7. So not very helpful to anyone. And again, following this principle of accessibility can benefit everyone -- I'm sure if you have been to one of our events in the past, you might have wanted to magnify the presentation screen.

So if one of these objects could have had a label like... Slides or presentation, we would have been able to explain that to people who attended our events and say... Okay. Open the objects list.

Click on presentation. And now the presentation will be full screen in your view. But when you don't have good names, or you don't have any information in the names, now you have to say things like... Click the second item in the list.

And this is being charitable here, with only two objects displayed in the objects list. It's very easy to have... Ten objects, 20 objects. So this is something where we need to have this mechanism of just labeling an object, just purely in the UI of Mozilla Hubs for any user.

But obviously the topic today is thinking and considering people who are blind and low vision, even if you can click on the different objects, and then visually look at them, someone who is using a screen reader needs to have more information supplied, to know what that object is. Because... There's no information supplied here. So...

This is an example of a 3D asset. This is our moon. That we can see usually from Earth. And the moon is in this slide...

I'm showing a 3D object that's rotating and showing different sides of the moon. So this is one of the concepts of spatial information, is that we're not looking at a flat image. We're not necessarily always looking at the same presentation of the image. Either the object can rotate in front of us, like the moon is rotating, or in virtual reality, we can walk around the object or move our self around it, and see different sides of the object.

So we do have a new challenge/opportunity in how we describe these objects. But even starting at a base principle, we need to be able to say: It's the moon. Right? Like an alt text attribute on the web.

You would be able to say alt=moon. So this is always where I start in what I'm advocating for virtual reality accessibility. Is: Let's start with having at least the same baseline support that we have on the web. To start.

And then let's add on top of that and innovate and make even better solutions for this new technology world. Yeah. So this is -- the file format on the web -- you might be familiar with GIF, a GIF or JPG or PNG. The typical file format right now for VR is called GLTF.

And so there's an opportunity there for the GLTF specification to have accessibility information put into it. And that's still something that's not actually active in the current specification. But I would say it would be great if we could get that built into it. Because it is such a widely used format, when we build spatial experiences.

So now let's switch into why I'm excited for today's presentation. We've been making progress, and we... At Equal Entry, hired an intern, a software engineer intern, Owen Wang, who may be here with us today. Owen lives in New Zealand, and Owen worked for three months on figuring out ways to add the ability to label and name objects in Mozilla Hubs.

And this is something that you can actually try today in a custom version of Mozilla Hubs. So Owen did an amazing job. And hopefully what you can see today is that one person in three months sort of dedicated on this accessibility idea was able to achieve quite a lot. And if we think about the resources that Meta, Microsoft, these large companies that are building these experiences...

If they can dedicate people to these tasks, I think we can see a lot of progress in this space very quickly. So what I'm showing on the screen is a pop-up modal dialogue that says "describe this object". So this is not part of the default Mozilla Hubs. But we added the ability to both name an object and describe an object into the Mozilla Hubs interface.

And this is only possible because Mozilla is an Open Source or shared source company. And that's why we chose to work with them on the project that we're demonstrating today. It's amazing when you can collaborate and take advantage of many developers' work. We're able to get a social platform that works and that people can collaborate in, and add accessibility features on top of that.

So that's what I'll be showing you today. And the end goal of the work that we're doing on this project is to hopefully work with the Mozilla team to integrate it back into the product, so that everyone can benefit it. And if you join a Mozilla Hubs event in any other locale or any other place, you should have the same ability to do this functionality. So I'm gonna show a different example of a 3D object.

So here from... Since I'm living in Japan, I chose to show a shiba inu. A dog. Popular in Japan.

And this is an example of searching for a 3D asset, and including it in a world. So this tool is called Sketchfab. And Sketchfab is similar to Corbis or Google Image Search. It's a way to search, though, only for 3D objects that you want to include in your experiences.

And some of these 3D objects cost money. And some of the 3D objects are free to use. So one of my first principles about alt text and why it's already sort of available is: If there's millions of 3D objects that could be included in a world, they already have labels. You know?

We're having to search this tool to find dog objects. And I'll show you quickly... I'm gonna switch to a screen of Sketchfab. Yeah.

We could choose instead of a dog... Let's search for cat. Just to be equal. So I want to put a cat into my 3D world.

Sketchfab lets me search on that term. And for the most part, everything that I see in the list has some relationship to a cat. Maybe some are less cat-like than others. But...

In Sketchfab, we have this idea of a label. And so this really simple object here... It's labeled "cat". It's a 3D model.

And the person who created it, Suushimi, labeled that. And so that way, when someone else wanted to try to find the cat, this label is already there. So... We wanted to make that more seamless.

And this is something that we definitely want to continue advocating with Sketchfab and other organizations that publish these assets. But if I show you my Mozilla Hubs world, which is on hubs.Mozilla.com, so this is where we usually host our event... If I add a 3D object into this world, I click on place. 3D model.

And search for cat. And now I have this cat that exists in 3D. I can rotate the cat. I can increase the size of the cat or decrease the size of the cat.

But it exists in 3D space. And now it's part of our world. So if I look at the objects list, you can see that the name, there's this reticulum for our logo, there's a reticulum for this art that we have of a mascot for our event. And then we can see that this is just labeled Sketchfab 70a2.

So there's no way for someone who is blind or low vision or who can't see the cat to know that that's a cat in the world. So that was the first thing we started with. In the work from Owen Wang. Was, in our version of Hubs, which is called Equal Entry Hub.com, I've created these different worlds.

This world is called Empty Space. There's nothing in it. So when I first join the space, I look at the objects. There's no objects in the room.

Use the place menu to add objects. So the UI is slightly different between hubs.Mozilla.com and hubscloud is the version I'm using here. But if I click place... 3D model...

I search cat... All right. Now it's the first cat in the list. Click on that.

When I pull that in now, in our version of it, the label is cat. And we're trying to follow this idea of ARIA, of role attribute. Or WCAG. To say that there's a role of 3D model.

For right now. So it's a cat. So it's a 3D model. And so that's the first part that I think is really powerful.

And I think... If you've had experience of the web, you can relate to the idea that most authors are not going to label objects unless they care about accessibility. So what we wanted to do with this design was consider that a lot of people are going to be finding these objects from a marketplace, like Sketchfab, so if we can take the label that already existed on Sketchfab, and bring it into Hubs, it's at least going to give that object more information than what reticulum.io...

a372 is. So I think for most of the time, this implementation at least goes farther than what... It always goes farther than what you would get by default, if there is no name. And then what we added also is the functionality to then adapt or improve what comes in.

So for people like us who are on this call, where we want to make sure that we're designing and developing really accessible experiences, we added these buttons into the UI, where we click on the 3D model for cat. And click rename. So I could call this... Cartoon Cat.

Again, I'm gonna caveat everything that... What we actually put in the labels for these controls is just as complicated as it is on the web. It's going to matter... The context, things like that.

But we're starting with making sure that we have the functionality that if we don't like the label for an object, we can improve it. So I could say cartoon cat. And now my objects list will say cartoon cat. And I can also add descriptions.

And so in this place... Describe. So right now, what we have is... I would call this kind of like ARIA-described-by, from the web.

Where we can write something longer. And then we also add this skill called custom, which we're hoping to use in our continued work. So, like, maybe I'm just gonna give a simple example... For description...

The cat is orange and fluffy. With a large tail. I'm not the best describer. But we'll say that's the description.

And maybe in custom, we would say... Size... Colon... Or height...

Colon... One meter. Weight... Colon...

5 kilograms. We wanted to play around with... By building this prototype, we can explore: What do people need when they're in the spatial world? Maybe it does matter if the cat is...

You know, 20 feet tall. Or one foot tall. Or if the cat weighed 200 pounds, or 10 pounds. [inaudible] here in Japan.

But the idea is that the scenarios can be so broad in VR that we're not trying to get overly prescriptive of everything that needs to be put into these values. But we want to create a mechanism to explore it and then work with people -- do real user tests and find out what's helpful. What's not helpful. And I'll just give one more example before I go back to my slides.

If I put in a different 3D model, like... This goes the same way that we have the same problem on the web... I mean, if someone doesn't give it a great name -- this one is called Toon Cat FREE. I don't think we want to name...

Toon Cat FREE is probably not a great name. It's not... It was more about marketing it, I guess, and putting free in large letters, to attract people's attention in the marketplace. So there is obviously not a 100% -- let's rely on what the author supplies.

But still, I would rather hear somewhere in there that it said it's a cat. And without this information, you don't get any of that. You just hear "reticulum" in this particular instance. And in a lot of other platforms, you don't hear anything.

So I could fix that with the same functionality. I could say... I don't like Toon Cat FREE. I just want to name it Cat.

And I can rename it inside of the world. Okay. So back to the presentation. I also want to say -- I can see there's a lot of chat.

I will be doing the questions and answers at the end of the presentation. So... Sorry if I'm missing anything temporal. But we'll get to all of them at the end.

This is showing just another example of what you can see in the user interface we defined, if you look at the object -- it's sort of how you see it in maybe PowerPoint or some other tool that has some accessibility features. We can tell you: This object has no description yet. Maybe it's a prompt for someone to go in and add a description. We haven't decided or implemented anything that says: Oh, you must provide a description for every object that you add into a virtual world.

But, you know, that's all possible. And those are all things that are open for discussion. And then I'm just gonna put this up, and not spend a lot of time on it. But part of the work that we're gonna be doing with our research group is: In virtual worlds, it's possible to have so many different contexts.

So if you start with... We only have two properties. A name and a description. Maybe the name Shiba is not a good best practice, because how does someone know...

You know, Shiba assumes that someone knows that's a dog. And that's maybe a big assumption. Whereas if you see it visually, in this context, you know... Okay.

It's a dog. Oh, and it's called Shiba. But the description -- tan-colored fluffy puppy -- this is just going with adding tan or like a color description, a texture description, and maybe like... An age description.

A different way we could do this is say: It's a dog. And then the description is: Shiba inu wearing red collar. Another example is: It's a dog. It's a white and tan shiba inu puppy wearing a red collar in a seated position.

So you know, it's always gonna be this complexity, which we deal with on the web, in a flat way. And now we have to think about this even more in virtual reality, like this dog can animate, it can change its position, maybe it does matter that it's seated, maybe the point of the experience is: You're training a dog to be a good dog, so you want the dog to be seated, or you want the dog to be jumping or shaking your hand.

So this interface -- obviously there's a lot of things to explore. And I just want to start from the idea that right now we have nothing. And there's no way to get any information. And we would like to start with simple things to provide the information.

Start working with people who are blind and low vision, to try out how these work. And advance standards and advance how we solve these accessibility issues in the future, from actually talking to real people that need to use these experiences.

Next is just the idea that in VR, pretty much if you're doing... Well, yeah, if you're designing any VR experience, you start with a world or an environment, and I like to take this back to web design, because I'm always trying to draw the parallels. But most of the time, I think of the world or the environment as the H1 of virtual reality. So in some virtual reality experiences, if you're going to be in an empty space, there's nothing in it...

And in different ones, you're going to have tons of objects when you load it up. In the same way you could go to a really simple website, like Google's homepage. There's barely anything on it. Or you could go to a Wikipedia article that's really long.

There's tons of subnestings and it's very complicated. So I think... At least considering that metaphor, one of the things you have to start from in VR is: Most of the time in social VR, you start inside of a specific environment. And you often can choose between many different environments.

So we want to have this idea of a room name and a room description, so this one... I'm calling it empty space. And this room contains a large open space comprising a grassy plain surrounded by mountains. That's just my starting point for what to do.

But the idea is that there is this H1 top level container for the world, and it needs to have some description. Because if you warp into a world and you're in a dungeon, or you're in a prairie, or you're in the mountains, obviously the starting point of your experience is important context to have. So we should kind of start from that idea that the room needs to be named. And then the next part, which is a whole other topic to really go deep on, but...

Your avatar is how you represent yourself in the virtual world. And there are lots of discussions about... Should the avatar be totally random? Have nothing to do with how you look in the real world?

Or should the avatar totally represent how you look in the real world? And depending on the context of the use of the VR experiences, sometimes we want to look exactly how we look in the real world, and we want people to recognize us. And to give you an example, we did an event in Spatial a couple months ago, and I could recognize Mirabai and Joly from their avatars, because they texture mapped their faces, and I used to see them in person, in New York City.

And being able to see them texture mapped in virtual reality -- I felt more connected, like... Oh, there's Joly and there's Mirabai. I could recognize them. And that was...

Yeah. For me, a unique experience. But that's obviously not a requirement. And in Mozilla Hubs, most of the time, we're working with...

I just put three simple avatars in our experience right now. But I just have this panda avatar, and again, I haven't labeled it well. But... Wouldn't you want to know that someone was walking up to you and was a panda, versus a tiger?

That's part of... If we do choose to represent ourselves differently, the same way I guess if we choose to represent ourselves similarly... You know, it's becoming more common now to describe what we look like in the real world to people in Zoom meetings. So this is the idea that however we choose to present ourselves as an avatar in virtual reality, that should have at least a name and description.

So in this case, the name would be panda. And the description would be simple black and white texture with the eyes and ears emphasized in black. So again, I'm not gonna comment on... Is that a good description?

But the concept of... It should have a short name and a longer description available, I think, stays consistent between rooms, objects, and avatars.

I want to call out just as a thank you -- I think Sage Freeman is here with us today, but we are including in the Equal Entry Hub some interesting avatars that we can use to do additional accessibility work. So these are interesting ones. Two of the bots or two of the avatars we've included is a partial occlusion and full occlusion bot. So full occlusion would mean that the screen is going to be completely black.

And so if you want to test some of the functionality we're gonna demo at the end part of this presentation, you can use full occlusion, and you will not see anything. So you'll have to rely on the interface that we've been developing. And partial occlusion looks like this. The image I'm showing on the screen is pretty much very difficult to see the objects in the room.

It's not trying to represent like an actual eye condition. But it's just trying to consider... If we're going to work on these implementations, can we create these avatars as we collaborate that let us not just rely on the visual sense? And so these are two bots that are created, and we'll have links to them that we'll be sharing out.

And I believe Sage Freeman is planning to build additional avatars as well. But we want to try these out. And I think it's really cool that we can at least try to innovate in this VR space for testing and evaluating designs. So then the next piece is this world where I'm starting to use labels.

With this, I'm just gonna do a live demo. Because... Frankly, I didn't have enough time to get great slides. So let's go to a demo.

So this is called Room With Microphone. And this idea -- which is pretty similar to our meetup events -- we have a room. It's fairly small. But we have a stage where someone would be presenting.

Then we have a microphone. Where someone might ask a question. To the speaker. And in the back of the room, there's a social area, where you could be discussing something not related to the speaker.

And this room is not perfect. Probably in some VR worlds, the socializing area would be outside of the room. But I just built this to give a pretty clear scenario, something that happens a lot in VR is -- you go to a presentation in VR, and then someone tells you... Oh, if you don't want to watch this presentation, and you want to go network and meet people, meet me out here.

And out here is sort of like the idea of like... Click here. It's somewhere else inside of the environment. And so if you're blind or low vision and can't see that, how do you go there?

So now this starts into new functionality that we added into Hubs, via the chat mechanism. So in Mozilla Hubs, there's this chat feature, and they have the ability to do different commands inside of the chat feature. And so what we have... I've documented all of these, and we'll share a link to this.

But /a11y, just like our meetup, can give you a poor contrast list of some commands that you can use inside of this world. But one of the most common things to do in the world is to say: Let me list what objects are in the world. So if I type /list o, I get this information... This room has the following objects.

Social, comma, label, 2-microphone label, 3-stage label, 4-... It's not labeled. So kind of similar to what we can see on the web. Sometimes there's something that's not labeled.

But when things are labeled, now we can do interesting things with them. So if I was inviting someone here in today's meetup, and saying... Hey, come up on the stage and present to our group... If you couldn't see the stage, we've added this feature to say: /move to the object 3.

Because stage is 3. And when I click that, now I'm up on the stage. I've moved to where that is. And I at least have a way to navigate this space, this virtual space, and get to a position, and if I'm giving the presentation, I'm on stage.

If I wanted to go talk to people and connect, if the host said: Okay. Well, for those of you all that want to not listen to the presentation, please go to the social area of the room. So in this one, I could say: /move o 1. Because it's social.

And now I would be in the part of the room where other people are chatting and connecting. So I've moved inside of the space. So that is just, I think, a very fundamental piece. That needs to exist.

Is... You can label spaces, and you can see too we're still using this idea of a role attribute. So the shiba inu earlier had a role attribute of 3D model. These all have role attributes of labels.

But maybe we would come up with something else as we work. But in Hubs, it's very easy to create a label. You just type... Let's say...

Whiteboard. I would type whiteboard in chat. Click on the magic wand icon. When I do that, it creates this object that I can then move and position in space on the whiteboard.

And so let's say we wanted someone to be able to move to the whiteboard. We say... List objects. Now whiteboard has been added as the fifth object.

I can do /move o 5, and now I've moved up to where the whiteboard is. So this is pretty cool, that it's that easy to label things in Hubs. And maybe we would want to have these be invisible for some users. But I think that's pretty exciting, that you can pretty quickly do that.

And enable that functionality. All right. And we're getting close to time. So I want to get faster here.

We've got a few more slides. So this is just a more annotated view of some of the commands. So this is showing a top-down view of the world that we're just in. I typed list /list o, and I found out that there's four objects in the room.

And one is this social area. Two is this element. Three is this element. Four is this element.

Then we've added a different command, which we support /fov, or slash view.

And the idea with this is: It's just inside of your viewpoint. So in the previous screen, everything was in the viewpoint, because we were sort of high above the world. But that wouldn't be realistic in the real world. Because we're not going to be flying up in the ceiling.

We're probably gonna be on the ground. So we're not gonna be able to see all four of these elements at the same time. As a sighted person. So fov is the idea that I want to just know what my avatar sees.

I want to have a list of objects that my avatar sees. And this I think is powerful to filter out... That there could be hundreds of objects in a world. And so this helps limit down how many objects are displayed.

And I want to close with this example of a school or a classroom. And so this came from Christian Van Meurs. I'm not sure if I'm saying that right. But he's a senior 3D artist at Mozilla.

And he built this pretty involved... I would say primary school classroom. And when you start inside the experience, you start in the hallway. And there are three rooms and various breakout rooms.

And so... Kind of hidden here, but this is labeled room one. The name of this environment is Hubs School V1. So the idea here is you are going to your first day of school and it's in virtual reality.

So maybe during COVID. Maybe some people here... Your kids did do this. But any day, even if you're going in the real world, the first day of school can be concerning.

Like, where is my classroom? How do I get there? Where is it located? So...

Also what are you gonna wear? Right? So the same idea that in the real world, you would choose the clothes you wear, in VR, you have avatars, and you choose how you present yourself. So thinking for people who are blind and low vision that want to both know where they're going and also want to choose how they appear, this kind of illustrates a little bit more realistically why we need this functionality.

Because maybe you have to go to your school in a VR environment. And so another question... Your first class of the day is English. How are you gonna find where the English class is?

And so this is a bird's eye view of the world, the school world. And again, what I really find fascinating working in VR, compared to the web, is: You have to think more like an architect, you have to think about the physical layout. You're basically building spaces. So this particular space that Christian built...

This is room one. This is room two. This is room three. And then room four is just...

There's five different breakout rooms. And this is a big hallway to navigate between them. So it's just like needing a map to figure out how to get to the class. So pretend room one was the English class.

You would need information on how to get to that room. So this is the English room. And this is, again, fascinating, that the room actually in the design of building this, and the tools that create this experience, you have to label these objects. So there's actually four tables, and the four tables have eight seats around each of them.

And there's also this reading area, which we'll ignore for now. But let's focus on these four tables in room one. Each table has eight seats. So...

If we had 20 people, 30 people, in the class, any of these seats could be filled. Any of these seats could be open. So we need to think about that too, for this scenario. It's like...

How do I know if the seat is open? How do I know if it's filled? And what if I want to sit next to my friend? How do I find my friend?

You know? If their avatar name is Stacy, I want to move to Stacy... Stacy is in seat two. I want to be next to Stacy.

All of that should be possible for someone blind and low vision. And all of that is possible to be made accessible. So this is showing a screenshot or an animation of Mozilla Spoke. This is actually how this world was designed.

And I just find this fascinating. It's already designed to have the tablegroups. So in the hierarchy of designing this world, Christian made four tablegroups. And then if you go into a tablegroup, there's going to be eight seats inside of that tablegroup.

So these are the four tables again. Just from a different view. And this is clicking through the different chairs. So...

Maybe they're not completely in order. But there is this... It's very similar to the DOM in HTML. But this is just a VR hierarchy of objects.

Here is chair one. Chair two. Chair three. So all of that already exists when we design the space.

And that should come through, I think, to be available for screen reader and low vision users, or people with mobility impairments, to navigate to these items. It would be great to say: I enter the room. There's four tables. Okay.

Tell me about table one. Well, there's eight seats and three of the seats have people in them. Or five of the seats have people in them. And you can kind of see how this would work with this tag.

Lastly, just one last concern. The same way that in the web you can create a really detailed infographic in Photoshop or InDesign and save it as an image, you can do the same thing in VR. So this particular poster is in room one of the school. Reading is a Passport to Countless Adventures.

Mary Pope Osborne. If that was important for someone to know and not just background information, if it's sort of burnt into the image or set as background, it won't be available for someone to discover. So that's another... Whole other area for us to go down.

We should have the ability to label these objects, if they need to be important. We should mark them as important. I want to end with saying thank you to Jesse Anderson. He's here also with us, Technology, Video Game, and a VR Accessibility Advocate/consultant.

He's done a huge walkthrough of this space. And everything I've focused on was really in this limited scenario of the 3D objects. But his video really talks about... What about logging in?

What about choosing your avatar? What about wanting to place the 3D model? Or using other features of Hubs? And all of that is important too.

And so we have links to his video. I really recommend that. This is also an Open Source project. So this will be more explained through our work in XR Access.

But this is just a screenshot of Visual Studio. And this is showing me editing the readme file of our GitHub page. And anyone else here that joins or anyone else in our group can edit this GitHub page. And I want to end with saying: Dylan Fox, who has been a regular also to our events, he's been really the spearhead for me, of working with XR Access, but all the work that I showed you today -- we're continuing to work on in the XR Access Working Group and ADXR, and we want to invite you to join us there.

Part of the reason too of showing everything today, if you have a passion in this, and you want to contribute to it, it's Open Source. We want your contributions. We're looking for volunteers. And we're gonna be doing a bigger presentation on this same topic in June at the XR Access Symposium.

Yeah. So I know we don't have too much time left. But let's do some questions. I'm gonna check my chat.

Meryl Evans. Thank you also to Meryl Evans, who is also one of our hostesses with the mostesses. She's been organizing both a11yNYC and a11yVR. And she sent me some of your questions.

I'm just pulling them up here. Makoto asks: Is the object list the same for everyone? So yes, the object list is the same for everyone. And part of the way that that's managed in Hubs is that most people don't have permissions to change the name.

So that's something we were... I guess we're not gonna have time to do the interactive demo today. But... You would need to be an administrator of the world to change these descriptions.

And so anyone else, like attendees, would just get what was set for the name and description. And there are some problems if Makoto and I were both an administrator, and editing the same object at the same time, you can run into conflicts with that. But I hope ideally... You know, you would say...

Let's get this world accessible before we host our event. You would be doing it in realtime. But there are scenarios where you would need to do it in realtime. So I do the objects list -- every prop and model in the scene are just interactive ones.

So yeah. I'm gonna pull up another thing that I didn't get to show. But quickly... IDing every object.

This is a big problem in how complex these worlds can be. So this is the school that I showed you all. As the final demo. This is Spoke.

Which is the design tool for building that world. So this is... You can see it's very organized. So Christian has it like...

There's ceilings. There's a hallway. There's room one. Room two.

Room three. If I click on room one, and Zoom in, room one has a floor. I can turn that off and on. So everything can be like...

Hidden or shown. But then when you get into the tables, you see that there's eight seats inside of each table group. But what you don't see... And this is, I think, the most complicated part...

I was showing like... This particular sign, Reading is a Passport to Countless Adventures. That is not a single clickable object. So the same way that...

Like, an infographic might condense a bunch of information to a single file, you can do the same thing in VR. And I think that's... Maybe that's a good thing. Maybe we don't want 100 objects described.

Because you can see how that would be overwhelming too, to say... You can walk into this room. You just want to know it's an English class. You don't want to hear reading is a passport.

Oh, and there's another one. Believe in Yourself. There's another poster... So the same way on the web that you have to mark things decorative, there's sort of a similar idea in VR.

Where by default, if it's just a texture and mapped onto the whole surface, it is background and it won't be announced. And then if you wanted it to be important, I think we could have the same idea as what you have in iOS, where you say: No, actually, even if this wasn't something that you thought should be labeled, I want this to be an accessibility element and labeled. All right. And next...

Claudio, if you add a description to the metadata, it assumes you'll want the same description every time. Right? Yeah. So that is also part of the design for the avatar and the room.

It stays the same every time. Because the assumption is that when you choose an avatar, and that's what you want to use, if you write it once and you're like... Oh, I want to use this avatar in other rooms... You don't want to have to keep rewriting it.

For the objects, there's this idea of... Which is another big principle, but... Any time... So Hubs...

These worlds exist only via hyperlink URL. And what happens is if you don't pin an object in the world, it doesn't exist. So just to give a really quick example... If I added the cat into this Hubs world, so we'll choose our cat...

Okay. So even if I added a description for this cat, if I don't pin it, so this is me hovering on the cat, holding down spacebar, and clicking on a pin, pin means that even if I close this URL and come back, I want that to still be there. So that's how the descriptions work. And our design is: If you pin the object, you added the description.

If you leave and then you come back to this URL later, that description should persist. And then... If you don't pin it, the object deletes itself when you close. Because if you created it and you didn't pin it, the description won't be saved.

And you would have to rewrite it for that instance. If these are 3D models, shouldn't they be treated as objects and perhaps each has a description like nutritional value on packaged products? Yes. So this is like...

I think what's so complicated. VR just multiplies the alt text issue. That I think it's very context-dependent. Because if I walk into...

Let's just take this room. I think this room has... Yeah. A periodic table of the elements.

If this room was, like, really used to be... You have to interact with the periodic table of elements... The teacher is like... Tell me what 21 is...

You're supposed to be like... Scandium! And this is the only way you can get that information, that needs to be accessible. But if it's like...

The teacher is talking about some totally different topic, nothing to do with that, and this is almost background information, you don't want all of that info in there. So it is very complicated. And I think to your example... It might be important to know every single object on a shelf.

It might be like... The shelf has artisanal design objects, and it doesn't really matter specifically what's there. David: Is there scope to work with Sketchfab for inclusion of alt description, and can this be imported in the platform for Hubs? David, I wanted to get that set up as an advocacy link for today's meetup.

But I will be following up with that. I don't know if there's an existing feature request at Sketchfab. If there is, then I'll link to that one. If there's not, then I want to create one, and say...

We shouldn't have to hack it. So basically what we're showing you, where we pulled the name in, was a hack. Sketchfab should provide that in a format. Yeah.

Do traditional screen readers like JAWS announce what's happening in VR space? Or do you need a new screen reader for VR? So that's also... I didn't get to demo that.

But... Maybe you need a new one. But what we tried to do in our instance was... If I turn on VoiceOver...

It kind of worked pretty elegantly in our design, so... If you stay... If I say list objects in the chat... Not a very good accessible room.

But basically what gets put in the chat can be read to the screen reader. So we tested that with VoiceOver and NVDA. And that's what we're planning to try to do, more of this user experience testing with people who are blind. It's not ideal.

It's not the solution we want. But it does work with today's screen readers. And so we do have a way to say: Run this command. Try it.

Give us feedback. And we're trying to prioritize making sure it did work. I want to thank everyone for coming. And make sure to connect with myself at...

I'm gonna go back to the top. @TechThomas either on LinkedIn or Twitter. And also, as I mentioned at the end, XR Access is the group that we're doing all this work, continuing on. So if you're really into this topic, and you want to help shape it in the future, there's gonna be a big conference in June.

And we're looking to get people that want to write code, test, be users for these experiences. Please reach out to myself or Dylan or anyone at XR Access about that. And you'll be able to collaborate with us.