Christian Cousquer has worked in digital accessibility for 17 years. He’s an accessibility and immersive technology consultant in his company called Ilargia. He’s an information and communications (ICT) expert at the National Conservatory of Arts and Crafts in Paris. It’s a major public higher education institution specializing in lifelong higher vocational training.
It was founded by Grégoire Abbot. They have a museum in the university where you can find a replica of the Statue of Liberty and splendid rocket engines.
What Is Virtual Twin for Digital Learning?
They have two mottos. The first one is they must enlighten the ignorance that doesn’t know and the poverty that doesn’t have the means to know. And then the Latin one: Teaching everyone everywhere. This is the very heart of the institution. They insist on the two terms “everyone” and “everywhere.”
What is a virtual digital twin for learning? An accessible digital twin in virtual reality (VR) is like sending astronauts to the moon and creating a lunar base. They have a chemistry lab in virtual reality. You can grab everything in the lab. You can do experiments, heating experiments, liquid extractions, rotary operation, and do a lot of training. There are about three and a half hours of different experiments.
You can prepare for danger by learning how to use an eye washer. You also learn the ability to select the right personal protective equipment (PPE), and you do small gestures like a thin layer chromograph.
Virtual reality is a powerful teaching medium. VR can offer a more visceral experience than traditional screen-based media. In a sense, it gives the user a sense of skill and, in many cases, it has an emotional amplifier and a memory of the experience. It taps another brain region, the live experience region of the brain.
You say, “I had an immersive experience.”
You don’t say, “I had an educational video.”
It taps another brain region, the live experience region of the brain. This interactive component has advantages and disadvantages. It strikes a balance between immersion, accessibility, and experience, which is the main challenge of virtual reality design.
You need to differentiate between virtual and immersive practical sessions. A virtual practical session can be a 3D animation on-screen. An immersive practical session uses virtual reality headsets. It’s an active pedagogy.
The learner — or spect-actor — and the virtual reality create a sensation of being transported to another world.
But in immersive learning, there is a hidden word. Immersive refers to immersion plus body-in-action. It gives the learner an active presence. In learning, you have an apprenticeship of skills, know-how, soft skills, plus feedback. It gives you learning trends.
Between the learning action and the feedback, there is the interactivity. It’s because the learner is in action that the computer, the simulation, is answering him with feedback. Because there is feedback, it varies in the learner’s response to feedback by action.
Learning traces give you learning analytics. We have a learning management system for VR. It can evaluate the progression of the learner.
How to Design
How do you design? Focus on your plan. You need to define a strategy. Take a baby steps approach. Each stage of your project must benefit from the previous stage of your project. Both from an educational and a project management point of view.
For example, for the chemistry lab, they have two partners. One company for the development called Mimbus, a virtual reality development company. They also have the Immersive Learning Lab, which is a lab dedicated to immersive learning. They have a partnership with them to acculturate the teacher to VR.
They have a starting phase and a requirement phase. They know we want to have a realistic environment, so they begin very early with the development of a 3D computer graphic. After that, they have an acculturation to VR for the teachers. Then, they have screenwriting of the first virtual reality module.
Every Digital Twin educational project has its pedagogical objective and specificity. Their technical management phase. The focus of Mimbus Chemistry is on learning professional skills, hand gesture skills, and processes involved in chemistry experiments.
The specifications are simple. They want a tutorial for learning how to use the VR controller. The learner needs to learn skills, hand gestures, the processes involving chemistry experiments, and how to use equipment in a chemistry lab. They focus on the teaching scenario followed by the technical.
The interactivity requirement will be fundamental. They want to create a realistic environment with full internationalization in French and English. And of course, they will focus on usability and accessibility whenever possible.
The simulation has three chapters along with a tutorial and virtual tour. The three chapters are Safety, Chemistry Experiment, and Digital Twin. The digital twin chapter will have a sensor on equipment in the lab that will send data to the simulation afterward.
Something important from the project management point of view is not to use a traditional top-down organizational structure. Instead, they organize themselves into “teams of teams.” The team consists of about 25 teachers and engineers, developers, project managers, and structural designers. They want a collective intelligence. The definition of collective intelligence is that it’s a learning organization where they learn from each other.
In a traditional top-down structure or a command of teams, there are problems in communication between those groups and those groups. This kind of organization fits very well with virtual reality development.
You can have the best teacher in the world. It goes without saying that if they don’t put a virtual headset on — if they’re not familiar with virtual reality — they will not be able to produce anything in VR. That’s why a VR project is a writing process.
On the other hand, VR content producers often have no expertise in educational scripting or the profession of knowledge and skill acquisition. That’s why everything must be formalized in writing. As an example, Mimbus Chemistry specification is about 300 pages because it deals with scenarios, scenography, storyboarding, and design-thinking.
A content expert must know a few things. They need to identify the impact of virtual reality in training. They need to understand the nature of the immersive medium well. They need to understand the physiological and psychological mechanisms on which virtual reality is based and which underpin its effectiveness.
Stage one: Defining the virtual environment
A rocket goes through multiple stages. The same applies to a virtual reality project. The first stage is defining the virtual environment. It’s like in a game. You need to let the user discover the lab. This means creating a virtual tour model with tool time and addressing the danger involved with some equipment.
Next is another module for the identification and characterization of the environmental equipment. It’s like a treasure hunt. It’s more difficult. You tell your learners that they have discovered the lab in the first module. Now, they need to go into the lab and identify and characterize some equipment in it. Many teachers make mistakes in this module as it’s a difficult one.
Stage two: Make small reusable modules
The second stage is to make small, reusable modules. We have made a small safety procedures learning module with virtual endangerments. One of the strengths of virtual reality is its ability to reproduce dangerous situations safely.
So, to do that, we break down the procedure into step-by-step scenarios with branches. One scenario could be facing the danger of fire in the lab.
The team consists of developers, structural designers, and chemistry teachers who don’t know development. The project requires finding a common language among everyone for technique, content expertise, and pedagogy. Teachers don’t know development. Developers don’t know chemistry.
The team uses diagrams where there’s a scenario for every action broken down into unit tasks. Then, after the user does that, they need to go into the lab and complete the module. Everyone on the team can read and understand the diagrams. The developer can turn the diagrams into code.
Stage three: Make modules based on reusing previous models
They can make more complex modules of chemical experiments by replaying the small safety procedure modules at any time. For example, a user is doing a liquid extraction, but he is making a mistake. There is a glass that falls on the ground and breaks. You can replay the safety scenario because it’s already been developed.
During these processes, there are interactivity challenges for solving expected problems and addressing accessibility.
However, do not try to reproduce reality. You need to find an interactive reality metaphor while always thinking about usability and accessibility. To do that, you need to work with simple questions. How do you empty your collected content from a container with a spatula when you have VR controllers in your hands? How do you wash a dirty spatula? How do you deal with flexible hoses in VR?
Here’s a tip. When something is done in two seconds in real life, it must be done in two or three seconds in virtual reality. Let me show you how we use the joystick to empty or collect content in the lab very easily and very quickly. Every piece of equipment in the VR is functional.
Accessibility in Virtual Reality
Accessibility means enabling people with disability to fully enjoy their rights and fundamental freedoms by removing any barriers that they may encounter. Digital accessibility in extended reality (XR) is a field of reach and part of this equality approach. It’s a combination of web accessibility and video game accessibility. Each has its specifics.
W3C XR Accessibility User Requirements lists user needs and requirements for people with disabilities when they use immersive environments, virtual reality, augmented reality, mixed reality, and other technologies. The BBC XR Barriers Research documents the barriers in XR environments. XR Access has also done a tremendous amount of work in XR accessibility.
Tutorials
What’s the way to address accessibility? The app provides a tutorial to users to help users learn how to interact with virtual reality. The tutorial teaches commands and the different metaphors used in the application. The tutorial takes place in a neutral and clutter-free environment to prevent distractions.
The tutorial is presented in sequential-themed rooms. The user moves to the next room after completing the activities. They learn how to move around, grab an object, move an object, and interact with an object.
The tutorial should also have different levels of difficulty, such as beginner, advanced, and expert modes. Beginners receive a lot of instruction while experts only get one instruction, such as “Do the experiment.”
Addressing different disabilities
A person who is deaf or hard of hearing can view the voiceover on a wrist-mounted tablet. The user picks up their wrist and puts the tablet in their view. They’re able to see the transcript for the spoken instructions.
The team opted to use a wrist-mounted tablet for two reasons. They wanted to avoid breaking the feeling of being in an immersive virtual environment. They also wanted to ensure color contrast between the background of the text and the color of the text.
For mobility disabilities, it’s important to test all the chemistry modules with someone who uses a wheelchair. For example, if an object falls on the ground, can the person pick it up without bending down? The magic grab laser makes it possible to grab dropped items without bending down.
However, addressing color blindness and dyslexia is more involved. The simulation has three possible actions with the hand controller. One is teleportation, which they associate with purple.
The second is to grab an object, which is associated with blue. And the third is to activate an object that’s associated with yellow. They chose to use blue and yellow because it allows someone with color blindness to distinguish the two colors.
They also found that about 10% of the students confuse their index finger with their middle finger. To help them associate the right color and the action with the right finger, they created “magic rings” to match the purple, blue, and yellow actions.
Color
There’s a parameter window that has slides for adjusting the theme, colorblindness options, and font. Users can filter color based on the type of color blindness, such as protanopia, deuteranopia, and tritanopia. Once they select a type, they can use the slider to change the intensity level. They can also change the font.
The environment avoids identifying anything by color alone. For example, a user selects items to answer a question. The wrong answers will appear in red (or whatever color it is for the colorblindness type) as well as with an “X” symbol. Color is not the only way to know if something is right or wrong. There are symbols to help identify right and wrong answers.
The lab also addresses the Proteus effect. This is a phenomenon in which the behavior of an individual in a virtual world is changed by the characteristics of their avatar. For example, a female user has male hands in virtual reality. The lab allows the user to choose between male and female hands.
Input preferences
The lab also allows users to modify their input preferences. They can remap the keys for controlling movement or interactions in the virtual settings. For example, they could hold down keys for an action. This can be changed to one click to grab and a second click to release.
Adding this type of functionality complicates the development. It requires modifying the entire experience including the tutorial. The main challenge in designing the virtual environment is the balance between immersion and user experience.
These accessibility settings have been built with open source on GitHub. People with disabilities helped build and test the settings.
Usability is also important. They have young kids testing every module. The purpose is to see if the kids can use the module without any help. This allows them to verify the ease of use and the difficulty level of understanding the lab exercises. They tested the lab with a 12-year-old gamer. It was the kid’s first time in virtual reality. They picked it up very quickly.
How Can Virtual Reality Be Used in Learning?
Virtual reality can be used in many disciplines and to develop many skills. If people need to learn professional gestures in technology, sciences, medicine, life sciences, or geology, they can do so in virtual reality with immersive simulations.
If people are learning soft skills in all teaching disciplines, they can do this through interviews and role-playing dealing with avatars. Virtual reality is good for immersive visualization of 3D calculations for math and sciences.
A virtual reality module can be put into virtually any existing curriculum. It can be in autonomy, upstream of a course, support for a course, after the course to reinforce knowledge, and as an evaluation.
Watch the Presentation
Speaker Bio
Christian Cousquer is an ICT Specialist in Web Software Engineering who is interested and passionate about web development languages, immersive technologies (XR), accessibility (a11y), internationalization (i18n), standards, open-source, diversity at the National Conservatory of Arts and Crafts in Paris.
Christian is in favor of a safe, open, diverse, and accessible Web. He’s an IT consultant who specializes in immersive technologies and digital accessibility in his company, Ilargia Inc.
Does Your Website or Technology Need an Accessibility Audit or VPAT?
Equal Entry has a rigorous process for identifying the most important issues your company needs to address. The process will help you address those quickly. We also help companies complete the VPAT to create their conformance reports so they can sell to the government or provide them to potential clients who require them. If you’d like to learn more about our services for auditing and creating VPAT conformance reports, please contact us.
