Designing Together: Co-Designing a VR Therapy | Accessibility NYC Meetup Recap

INDIRA: Hi. So I'm Indira. And I work at the Helen Hamlyn Center for Design in London, at the Royal College of Art, and for the last year and a half I've been working on a VR therapy project which includes working with people with the lived experience of psychosis in the co-design. The project is called gameChange and it's a VR therapy for people with psychosis to make them feel more confident in everyday situations. A recent audit recommended that people with psychosis and their families get some sort of psychological therapy. But when they did an audit of one of the NHS trusts in the UK, they found only 5% of people receiving it. So we're hoping that this VR therapy will make it more accessible and more available, initially, in the UK, but hopefully it can go worldwide. So I don't know if people know

what psychosis is. But it's a mental health problem that causes people to perceive or interpret things differently from those around them. So it might involve hallucinations or delusions. This could be really strong beliefs that aren't true, or it could be hearing voices, seeing things, or even sort of touch and smell hallucinations. And this can happen for a number of reasons. It can happen with schizophrenia, with bipolar disorder, it with happen with certain diseases like Alzheimer's, and due to sort of drug and alcohol. And people with a diagnosis of psychosis often find it difficult to go out, because they feel very anxious about being outside in social situations. This can be because they fear for their own safety, or they fear they might have an episode, and for the safety of others. They interpret

how people are looking at them, which could be threatening, so there's a lot of reasons people do get anxious. And this can lead to social isolation. Which again, in turn, leads to further medical problems or physical problems. So the gameChange project -- it was actually funded by the National Institute of Health Research in the UK for 4 million pounds. It's a three year project, currently in its second year, and at the moment, it's in clinical trials with 332 people. It's actually being led by the University of Oxford, along with the Oxford NHS trust, and as you can see, we've actually got quite a lot of partners working on it. I've been working on the design, the development side. So we've mainly been working with Oxford VR, who are actually building the application, with the McPin Foundation,

who are advocates for people in mental health, and with Oxford, but also with other groups in the partnership. So to let you know more about the project, there are four work packages. There's the design and development that I've been working on, but there's also looking at implementation and clinical adoption, establishing the clinical benefits of the VR treatment, and assessing the cost effectiveness. So this will happen over the whole thee years. To let you know a bit about the Helen Hamlyn Center for Design, I'm based in Battersea, in the Kensington part of London. It's one of the only postgraduate colleges in the world looking at art but also design as well. The Helen Hamlyn Center on this project -- we were leading work package one, which is design and development, partly because our involvement

from the outset around inclusive design. So the Center itself is about 30 years old, and was established on the ethos of inclusive design. And it came out of ideas that have been going on for sort of decades, but in the early '90s, people got these together, and sort of called it inclusive design. And one of the people involved in this, there was a conference, a symposium of applied ergonomics in the early '90s, and Roger Coleman talked about inclusive design for aging there. He was one of the co-founders of our Center.

And it was really the idea that there weren't very good facilities and good designs for older people, and how could they bring in young designers to work with older people, do co-design, to create better designs and better products? And this has grown over the years. So now there are areas in health care, which I'm working in, with a lot of work on neurodiversity, and sort of different areas, but it's all about inclusive design, including people in the design process. And we publish a lot of our findings as well, and make them available to people. So we try and keep things open. So we can sort of further knowledge.

And just a couple of other projects I thought might be interesting to talk about, that we've done. There's one called design and the mind. So this is the Wellcome Foundation in the UK, which has a hub which is trying to be inclusive and get people working and engaging. A colleague of mine, Natasha, did some work with them, to include neurodiverse people.

They wanted to improve their approach, how to work with people with neurodiversity. In engagement. And another one was How We See, which was working with the RNIB, and this was looking at inclusive design to help increase public understanding, attitudes towards eye health, through the experience of the sight loss community. So that's the URL for the Center, and also I put the URL underneath for -- we publish a yearbook. Online at the moment is the 2018 one, but it shows all the different projects that are going on in the Center. So it's kind an interesting way to see what we've been working on. Now, I'll go back to gameChange. And if I can, I'll see if I can show a video that sort of explains some gameChange a bit. Just a minute. Unplug.

[Music]

Sometimes people can't get out of the house to do something. They worry about it so much in themselves, before they go out that is that they feel they can't do it at all. So they avoid doing it. You'll be having a feeling in your chest all the time. It's a heavy weight on your chest, feel like avoiding to meet somebody or to go to see a GPO, or to go on the bus It's very hard.

I'd feel paranoid. I'd feel scared. It's about leaving the house. [Music]

It should be a good way to try out the things that you scared of, and I sort of like learned that It's not as scary as it actually is.

I would say try, and give it a go, and you can do it at your own pace. and you can change, you can stop when you like.

I've never done it before, so the first time I tried it I was like, whoa this is really good. It's like, it's fun, it's a fun thing to do. It's a fun concept. You put on the headset, and you can see, and you can hear all these different situations. So, you use the virtual reality to help you with your thoughts and feelings, and your confidence, and then when you go back into the real world you might feel a bit better.

Now, say you're in the shop situation when you start having some kind of worry, and then, I know you find a way within the VR to overcome that worry. And maybe, like, when you're in the real world you have the exact same worry, and like, "What was I doing the at the VR?" "Oh, I did such a thing." And then you can try in the real world because you've practiced it over and over again in the VR.

[Music]

gameChange has basically been shaped in every aspect by the experiences of people who have psychosis so, fundamentally, people with lived experience have been involved in every area of the study, and they've been enormously influential. There is a group which has basically selected each of the different scenarios which are used in gameChange, including the GP surgery, the streets, going to a pub, going to have a cup of coffee in a busy shopping center. Those have all been selected by people with lived experience, and indeed what people do in those environments has also been massively influenced by workshops which have been held, and meetings which have been held with people who've lived experience of psychosis. So, I think, people with lived experience - I've just been astounded, really by how much influence

they've had, and it's been a a pleasure to see just how quickly and how far things have come. [Music]

We're very excited about the gameChange VR Therapy It is an easy to use step-by-step program to help people be more confident in everyday situations, but it's important we test it in a clinical trial to find out whether it works as we hope. 432 patients across five centres in England will take part in the gameChange trial. Half of the people taking part will receive the VR therapy and half will not. We then see how the people who had the VR therapy get on compared with people who haven't had the VR therapy. Whether a person has the VR therapy is decided by chance. Everyone in the trial has an assessment the beginning which is then repeated after six weeks, and six months.

By the end of the gameChange trial we will know whether we have a powerful new treatment for use with patients across the NHS. [Music]

INDIRA: So gameChange is based on psychological principles. So we worked very closely with a psychologist at Oxford University. And they've done some work before, around this. So it's the idea that people with psychosis, to cope with going outside, will create defense behaviors, but these defense behaviors aren't necessarily very healthy. So it might be things like only going out at certain times of day, or looking down when you go out, not making eye contact, wearing dark glasses, wearing a hat, and not really engaging with people. So it means that their fears are sort of compounded, because they're not seeing anything else. So psychologists will often take people with psychosis out to places that make them feel anxious, and sort of get them to look around. Get them to reassess their fears of being there, and

hopefully they can then see that it is generally safe to be in those situations. It comes off the back of a sort of whole series of previous research. One that looked at automated psychological therapy, to check that that worked. And another which looked at the treatment -- virtual reality in the treatment of persecutory delusions. So the therapy itself, as it said in that video, it is a self-contained therapy. So a virtual coach takes the person through the therapy. There are six different environments they can choose. Which was mentioned in the video. And for each environment, there are five levels. There's five different tasks.

That don't get cognitively more complicated. But get harder to be in. So you can imagine if you're in the pub, it starts off and it's fairly quiet, and there's no one in there, or very few people in there, and the next level, it's more people. There's more sound.

As the levels build up, there's more people. More noises around. And you might have to then interact with someone who might cause you anxiety. We're expecting people to do six sessions in this, of about 30 minutes, in virtual reality. I think it could be up to 8. And there's always a person with them. So there's always a support worker throughout, while they're in the headset. It's not something people would do alone in the home. It's part of the therapy, where they would have support, non-virtual support. And there are advantages to this. I don't know how many people here have used VR, but we often respond to it as we do in real life. So it means if we've created these environments, people will respond to them as they would in real life, and feel those same anxieties. But it lets people try

things out in a safe space, and try out different ways to cope with that anxiety in this safe space. And they can repeat it as well. So if there's something particular that's worrying them, they can just repeat that particular task. So now I'm gonna talk about the design, the co-design. So as I said, we led work package one, the design and development, because of the history of our work with inclusive design and co-design. So it was really important for the development of this that it was -- I think it was the usability, it was engaging, and it was scalable. Those are the sorts of things that would make it a success. And so, for this, we worked with people with lived experience of psychosis. We worked with the psychologist. But we also worked with the developers as well. And it was sort of -- the

criteria for designs were: Did it meet the psychological principles of a therapy? Was it actually -- we could build it in VR, and we could build it in the time frame? We had a year to design and develop the VR for the clinical trial. And is it engaging? Is it relevant to people with psychosis? And it turned out to be quite an iterative process.

Which we didn't realize at the beginning, that that's how it would be. We were doing the designs and building the product sort of at the same time. Going through sort of multiple groups of people, to get to that point. And so it sort of built up slowly from around June 2018, and we delivered the designs in December 2018. So they could be built out by the developers. This is a work in progress diagram. I'm trying to work out the structure of how we did it, and I'm still... These arrows aren't quite right. When we first started, we knew we were going to have a lived experience advisory panel and we knew we were gonna have workshops, but the rest of the structure developed in the first month, really, as we got to know

what we needed and how it worked, and it ended up working quite well. So I'll go through. We created six groups, in the end, that all worked together and separately. So they could meet separately and then feed into each other, as groups. So the first one that we knew about was the lived experience advisory panel, and we're really lucky to work with them at Pin Foundation, one of the partners on this project. Because they put together this panel. And it meets -- it's running over all three years. It's working on all work packages. So it's people who really invested in the project, who know the project. So there's continuity across the whole lot. And they meet three to four times a year. And they also -- well, the first year, we met with them five times. And they worked on -- and also had access

to them remotely, through the McPin Foundation. So I sent a couple of questionnaires to get more information and more sort of feedback. You see the questionnaire here that I designed with a psychologist, asking what sort of characters people would find would cause them anxiety. We need to find out things like that. So that was really helpful. And then we ran the workshops as well. So this was -- we knew we were going to run six workshops. Five in the cities where we were running the clinical trials. And one in London. And these workshops would be people with lived experience of psychosis would come. We had quite a range of ages, of backgrounds, of male and female. Because we wanted to get as many people as possible involved in this.

And in the workshops, I'm sure most people here have been to workshops. There were lots of post-it notes. Any idea was fine. Nothing was wrong. And then we sort of gathered that all together. But then this was something we didn't plan for, was creating a working group. And this was partly because we had a design time shorter than we initially expected, and we wanted to get as as much input as possible from people with lived experience. So the working group was based in London and had a number of people with lived experience of psychosis, plus us as designers, the psychologists, and people from Oxford VR, the developers.

So while the workshops were more sort of -- anything went, this was far more: Can it be done, is it relevant, and does it match the psychological principles? So having all those people in the room together meant that any idea could be discussed by all the relevant people. Which meant we could really work through ideas faster. So when we had to get them signed off, we knew they had already been checked by people with lived experience, by the developers, as designers, we checked them, and with the psychologists. And there's an example of this.

One of the tasks was going to be catching balloons, and the developers said: That's actually quite hard to do in virtual reality. Because you have the physics of the balloon and the string looking real and the person catching it. And in the short development time, we knew we had to get rid of that idea. So that could happen really quickly. And then we had partners meetings. And you can see the partners here, who attended those. And this was where we presented the designs. We could discuss them. We could see what worked for everyone. What didn't work. Ideas could get signed off. The deciders on the project were the psychologists from Oxford University. So they would be presented with all the designs, but there was a lot of discussion. It wasn't like they said -- yes, we're having that.

No. There was a lot of discussion around the designs in those meetings. And the other part of the structure that I'm not gonna talk about in detail was the subgroup and the implementation team. And the subgroup really was -- there were three of us. I had met on the phone with someone from Oxford. One of the psychologists and one of the developers from the development company. And it was just a weekly phone call, where we could ask questions from each other, we could give information to them, that they could disseminate through their organization.

It was a quick way of getting messages back and forward between the different organizations. And also to talk about -- we had an implementation team, which wasn't the same people. But that was actually running through how the designs would work. And creating the design documentation.

So I'll go into this in more detail in a bit, but this is just on the design of the coach. To show how iterative it is. All the ins and outs and all the different meetings that happened to get the design of the coach and the script to the coach. So I'll go back into this in a minute. So we started with the fundamentals of the design. This was the psychological principles. Thinking about presents. All those sorts of things. So that's sort of our first thoughts, when we started work on this. And then we had first decisions. So this was...

One of them was the environment. Which again, I'm gonna talk about in a minute. How the lead worked on creating the environments, but it was also looking at the environments and putting them into a matrix to make sure there were psychological principles to match each environment. So the pub environment was about people being in a situation where they didn't know what would happen. So we needed to make sure that was ticked off. But they could also match other environments, like people being... Other psychological principles about people being close to them, being a problem, noise, that sort of atmosphere being a problem. So I don't know how many people here have run workshops and designed them.

I see a couple of nods. It was pretty much as you would run most workshops. But because people have had lived experience of psychosis, and we needed to be aware of that, there were a few things -- so we made sure there was always a quiet space people could go to. So we had the main area, and then a quiet space. The psychologists came with us to the workshops, who worked with people with psychosis regularly. So they could really describe the projects and talk about the psychological principles, as well as be there, if people were feeling anxious. And there was someone they could talk to about that. The agenda -- I would always have set aims for each workshop of what we're looking at. And I would do an agenda.

But I would keep it quite fluid, because you're never quite sure how much you would get through. How much... So you would have to think of maybe new ways to get the same thing you wanted. So it stayed fluid throughout. So quite a few... I created quite a lot of things, but didn't use them. But that was quite interesting. And then the other thing was: We had needed space for people to bring their family members or their support workers, if they wanted to, as well. They could be part of it. And everyone was given the opportunity to try virtual reality.

And they did all try it. But it really wasn't pushed on people. It was an option that they could do or not. And we were lucky that we had -- Oxford had already created a therapy for anxiety. So we could use that as an example. So people could see what a therapy looked like. And then we could talk about designing it. And a lot of the work we did was on sort of solo brainstorming, and then group work. So we could have that point where people who didn't feel confident speaking in public could get their thoughts across. Plus some of the discussions afterwards were really useful. So when it came to designing the environments, we worked in the workshops on this. And we also sent out a questionnaire. But it was

sort of... They couldn't be too nice. They couldn't be too nasty. It was sort of a general environment that you would see anywhere. And again, it couldn't be too London-centric.

And with the character design, it was having characters who could trigger anxiety. But also just how many characters in that space would be a problem. And how close the sort of proximity of people -- how do you build that up? With the task design, again, we did a questionnaire. But also, we did one workshop looking at the cafe, where people put post-it notes on different areas of the cafe, things that would happen in the cafe, that would cause them anxiety and sort of trigger problems. And what was interesting in the process -- some ideas that came out. We would look at why those did, so we could maybe then change them to other environments if they didn't work in the environment they initially were talking about. And we did a lot of work on the script as well. This was partly looking at the language

the coach would use. So it wasn't patronizing. That word came up a lot. We don't want it to be patronizing. Words like "well done" didn't always work. But it was looking at how people wanted to give encouragement. How they would give encouragement to their friends.

If they were doing it. Getting quite natural language. And so I'm just gonna give a couple examples in more detail about how things were created. So the environments were done in a workshop early on, with the lived experience advisory panel. And they looked at what caused them anxiety. And also the social situations they wanted to be in, but weren't confident to be in at the moment. And then sort of voted on a top ten list of those. From that, the psychologists and the developers looked at it and thought: We can't always develop everything everyone wants. So how can we create environments that fulfill these things in the top ten?

So the six environments that were chosen, while not necessarily always matching word-for-word what people had asked for, they fitted with what people wanted. And the next example I'm gonna talk about is the coach design. So we did a workshop where we looked at what people would want in a virtual coach. What would they find supportive. And also the developers wanted to know if a coach could be... If it had to be a full body coach, or if it could be in an iPad-sized thing or if it could be voice only. So we did quite a lot of work on that in the workshops. The voice only went straight away. People who are hearing voices don't want a disembodied voice. It wouldn't work. It wouldn't be good. But then quite interestingly -- we went for the full bodied, because that was everyone's preference. But

there was an interesting one, where someone actually quite liked it in that sort of space, because it felt like they could call on the coach when they needed them. It was like a pop-up. Like having that phone. So it was interesting to get people's response. So I'm gonna go back to this diagram again. Of the coach. And how all the different meetings that went through it -- and the first thing I've put in here is things that aren't connected to the workshops. The design research, the desk research, the analysis we did from the workshops. And the prototypes. So that was all part of it. The design decisions. And the broad knowledge. And I think what was quite important for me, as a designer, you come in with your preconceptions of what you want to design. And I sort of put those and

wrote those down at the beginning, because I didn't... I think it would always bias what you do, and I didn't want to overly bias what was coming back. I wanted to make sure what my preconceptions were, so I could put those to one side and actually listen to what people were asking for and why they wanted it. So through June, July, and August, we had a series of meetings, and you can see the first one is an all-partner meeting. And each meeting, there's an aim. We had inputs from different places, and outputs. And you can see the inputs then -- the outputs become the inputs for future meetings and future workshops. You can see early on, it's a lot of partner meetings. Then we had a workshop, then another partner meeting, and then another workshop. It went back and forward like that, iterating through.

Through September and October, we really sort of... That's when we decided it was going to be the full body. Then I think it was in the Nottingham workshop in September and Manchester, we started to look at the script and what the coach would say, and that continued through November, and the partner meeting, and in December, we had a meeting with the Leap, where we talked about the script and also the coach. And made final decisions. So it was just before Christmas break, we handed in all the documents. All the design documents.

So for the implementation of this, I worked quite closely with Oxford VR, and with psychologists from University of Oxford. Once we knew what the tasks were, we had to map out the task flows to make sure that everything worked. A lot of this was done on paper. I just did drawings. To see where people would stand. And we also did use lucid charts to do flow diagrams, so we could see where it branched. If someone didn't complete a task, what would happen. But we also mapped out... So this was a mapping out of the bus. So we could actually see: What does it mean if someone takes a step in VR? Because we often don't know the space that this would be delivered in. So we needed to make sure that one step

wasn't too much. They were within a certain area. How far could someone get into the bus? And also, it's things like... The person has to pay when they get on the bus. Which hand is the card for them to pay going to be in? Is it gonna be there all the time? So we can just act out those things to see what worked. And we also created prototypes. I was gonna show a video. I won't now. But this is one of the prototypes of the bus. This is partly a game for us to work out what it would be like in that space. The sizes. We could go in there. But we could also take these to workshops. As it developed, we could start showing people what it was going to be like in gameChange. And then we created design

documentation. This is a sort of overhead animation of one of the tasks. But we also created design documents for developers that listed what happened in a task. The psychological principles for that task. What the coach was going to say. What other interactions with characters there would be. What objects needed to be picked up. What sounds. Important things.

So as the developers went through, if they needed to make a change, they could see the reasons why we put something in there, to see the importance of it. And if they had to make a change, it would match. It would match with the psychological principles. And then there was user testing. So it was sort of in the original plan that we had to sort of get 90% on usability. And engagement within it. So we started user testing, looking at first of all the glitches within the virtual reality application. But then it started just before the clinical trial started, the user testing, 90% of users did find it engaging and usable. So what's happening now? And what next? The clinical trials are taking place at these five cities in England. They are running... They'll finish later this year.

So hopefully around this time next year, we should get the results on how effective this virtual reality treatment is. The other work package is my work package. The design and development is pretty much done. So I'm now working on a paper, looking at sort of writing up what we did. But also, sort of looking at other research. But the other work packages are still looking at how this can be used within the NHS, and how the sort of road to that is. But what I found quite interesting... I was working, doing the design of people with lived experience of psychosis. Another organization, MindTech, they were working with people who were delivering it, running similar workshops with them, to check how they wanted it to be formed and how to deliver it. So it wasn't pushed on them at the end.

It was thought out, how they would deliver it, and that they were comfortable delivering this therapy. I don't know if people who run workshops -- often questions come out that are really interesting, that haven't been researched. And especially in virtual reality, there's a lot that hasn't been researched. So over this next year, I'm going to be looking at some of those questions that came out of the workshops. One of them was sort of how people can see how they're doing in a VR therapy, in VR. So how they can track their progress.

And you can find more information about the project at that URL. And it's also on Twitter. And thank you. (applause)

Thank you very much, Indira! Now we're gonna open up the floor for questions from the audience. We've got like 15, 20 minutes. So go ahead. Who's got questions? Go ahead.

Stand up. Say your name. And speak into the mic.

Hi. I'm Johnny. I do user research myself. I'm very interested in how you were able to get that discovery phase, get that user, understanding the user needs in a way that did not invoke trauma. I can imagine how this could be a traumatic moment, to ask: What are these trigger points? How did you get that information and make it in a way that you would make sense of it?

INDIRA: It was very important that people felt comfortable in the workshops. What I also realized -- that people who have had psychosis were really invested in finding a treatment for this. Because they had been through it, and so they were quite keen to do this. So McPin Foundation sort of put out notices to find people. So they knew what was going to happen when they came. As we said, we put things in place, that people could leave at any time. People could bring their support workers. It was having that safe space. We also, if people didn't want to speak, we would work with them when they were writing, doing their brainstorming. So we could speak their words for them, if they didn't want to speak them themselves. So they could get heard. They could put up post-it

notes. So everything was captured. And having the psychologist there meant that there was someone they could talk to. Because virtual reality anyway can cause problems for people.

You can get motion sickness. There can be different things. But it was making sure that there were things in place, that if something was triggered, there was a support there for them. But actually, through the process, as I say, everyone did try it in the end. And I think everyone -- though not everyone liked it or thought it was good, none of them found it a negative experience.

Other questions?

Hi. My name is Joel. I'm a product designer, and I know that in app development, there's a lot of give and take. So I'm wondering if there are any things about gameChange that still keeps you up at night.

(laughter)

INDIRA: Not anymore. No. It's been a while. You do get quite invested in some of the ideas that can't be made. And when you're developing, particularly because it was such a short development time, things couldn't go in. But in a way, that's led to research. And one of the things that came out, that we couldn't put into VR, which was: How could people sort of record what they were going to do? We actually ended up doing an extra workshop, looking at the treatment outside VR, how it was delivered. And the materials, written materials, that people could do. So there's often a way, if you can't use it in technology, some of the work went into non-technology, and so creating a booklet where people could write their thoughts.

Some really interesting things came out of the workshop around that. Which has been a shame not to include. But we could include them in the end. Non-technology.

Hi. I'm Sonya. I'm a software engineer. So I was wondering... Given that the problem that you're trying to solve is very complex and broad, and the solution that you delivered is very detailed, down to where the user is gonna stand, were there any assumptions that you had going into the design or development phase? Which were later proven to be incorrect?

Or any edge cases or blind spots that in retrospect you realized that you had missed?

INDIRA: I think we'll know more at the end of the trials, if there's something we've really missed on. Unfortunately I keep saying... How are the trials going? And they can't tell us. Because it's a trial and they have to keep that secret. I think going in, I definitely had assumptions. Because you come in with a design background, and you're used to making it a certain way. And particularly VR comes a lot from gaming and those sort of thoughts.

But it's very slow. I don't know if anyone's tried a therapy VR. I tried a PTSD one as well. And they are very slow. It's not about sort of action. It's about people staying in a space. And them having to be there. And I suppose the other assumption is you have all these visual ideas that you want to implement. But actually, when you're working with people with psychosis, you have to be really clear what is meant to be there. Because with the hallucinations, it needs to be clear that this is what you're meant to see. So we had to make sure that we didn't use sort of colors or sounds in a different way. Because we were going to sort of use color to represent things, at different sort of points. But actually,

you need to keep it quite simple. Not simple, but quite realistic in how you do it.

Hi. I'm Stephanie. I'm a UX researcher. I have two quick questions. One: It might be the limitations of VR. I don't know a lot about VR. But it didn't look hyperrealistic.

But I wonder if there's a benefit to that, in making someone feel more safe in the space.

INDIRA: What's interesting in VR, and I don't know if this will change, as VR is used more and more, is... It's more important that it feels realistic than looks realistic. So it actually triggers your mind, even if it doesn't look that realistic. What's interesting with the gameChange therapy is it is actually a lot more realistic than a lot of other therapies.

But when we were doing the workshops, what people didn't like was when the movements were wrong. And the facial expressions were wrong. And they were far more important than, say, what the environment looked like. So I found that was quite interesting.

And then my second question was: Were there levels? You said people could track how they were doing. Do... How far you stand from me -- does that get closer as you progress?

INDIRA: Yes. So there's a cafe, where you're in a queue, going to order a coffee. So when you start, you're on your own, and there will be people behind you, and they're closer to you. So it is that sort of thing, where there will be more people, and they'll be closer to you, as it goes on.

Xian?

I want to say a quick hi to Thomas, who I know is creeping on the streaming. Definitely couldn't miss your wonderful presentation, Indira. So first of all, I want to thank you so much for sharing. I feel like I've learned a lot. I don't know about you guys. But anyway...

I am always curious, and I think first of all, the fact that this is sort of what I always tell developers, corporations, that they need to work with the populations in need closely, and it seems like you created so many layers of protection and really tried to embrace feedback. I'm curious: What was something that you were surprised by? Or that you would take into a future situation, that you learned? Obviously six months you had.

So I'm sure there were a lot of learnings. But I first want to commend you. I'm curious who came up with this. Was this your idea? Was this a group of people saying: This is the best way to do this?

INDIRA: It was actually... The head of my team came up with the idea of the working group. I think that was probably one of the things that for me was the best thing that came out of this, and I would definitely do again. We hadn't planned it at the beginning.

And it meant that people with lived experience of psychosis were involved throughout the design process. So it wasn't coming in at set points. And sort of testing things. Or even sort of asking for ideas. It was really -- it became that point where you're part of a team, all working together with a psychologist, and the developers. And we all got to know each other. We had the same people, often, coming every week. And I think you need that continuity. You need lots of different people. Because you need lots of different voices.

But also, having that one group, that is something I would definitely do again. And also, even just the weekly calls -- I know it sounds really small. But just having weekly calls between the developers, the designers, and the psychologists meant we could get questions answered really quickly, and they could go back to people in their organization who were quite hard to get ahold of, sometimes, and we could get information back. So I think those are the two things that for me worked well, that I didn't expect.

That's wonderful. Bravo.

Hello. I make 3D models like the ones that you saw there. I wanted to ask you something very specific as a patient. So, for example, there must have been some very generic levels for everybody, and there must have been some very specific levels. So I wanted to ask you if you could give us an example of each of them. For example, I don't mind being outside, but when I get into an elevator, and if there's no one, then I'm anxious. If there are ten people with me, then I'm anxious. I'm okay with two or three. So if I come to you with a specific issue like this, how would you tackle that?

INDIRA: Yeah. I think this is why we tried to -- with the six different environments and six scenarios, this is why when we worked with people with lived experience at the beginning, and they came up with ten, we tried to include those ten in the environments, to make sure that there would be an environment -- so people don't have to do all the environments. They can pick the one that they want to do, and the one that causes them anxiety. And that they want to be in. And I think that's quite important. Because at the moment, it isn't tailored to a specific person. But if it's the closeness of people in the cafe is a good one -- because you're having to stand there and you're having to order. There's a shop

where you have to go and buy things, and you have a list of things you have to buy. So that can cause anxiety. You've then got to queue, and there are people waiting in the queue behind you, and it's very slow. So we tried to create different things. So hopefully whatever someone's anxiety is, there will be something within it that would trigger that. And that they could work on. But I think... Because the nature of it at the moment...

There might be people who think that there isn't something there, but I would imagine for most people, there would be something within it, that would help them.

One in the back?

Hi. My name is Jen. I'm a UX designer. So I'm curious about... This is about accessibility.

But I'm curious about actual access. What about people who are neurologically diverse, but have a lower income level, or don't have... Their access to mental health care is not ideal. So is this... Do you have a contingency plan?

INDIRA: I suppose this is slightly different to the US. So this would be delivered by the National Health Service, so it's free at the point of entry to everyone. But as I said, there's only 5% of people getting psychological treatment now, and it isn't always easy to get mental health treatment within the NHS. You go to see your doctor. You might have to wait months to get treatment. So it isn't sort of all great over there, that you would get it straight away. And if you have money, of course, you can get it straight away. But with this treatment, we're hoping -- because it's relatively cheap, it doesn't have to be recreated for each individual person, it's in a headset, it would be peer support workers.

Because this actually came out with working with people with lived experience. They wanted peer support workers to deliver it. So they're recruiting peer support workers for that. So hopefully it means that this treatment will be... There will be this treatment for more people. Because at the moment, psychologists try to take people out, but again, it's quite limited, their time. So there shouldn't be a restriction on this going out to the NHS.

Though the people doing the cost evaluation are looking at that. I think that's why it's really important that we had these different work packages with different expertise, so they can actually see how it will work and what the costs would be for each NHS trust and how many people it could reach.

So a follow-up... Putting economics to the side, a major issue a lot of these people face is leaving their house. So they would have to leave their house to access VR equipment at a mental health care facility. Is there a contingency plan for that?

INDIRA: This is what I'm not quite sure. So MindTech is looking at how it will be delivered. As I was going to the workshops, they were talking about delivering it to people's homes or where they would deliver it. I'm not sure what the final decision was on that. Whether it could be taken into people's homes. But there are things around that, around... Would they have the space? Would they have the privacy to do it? Using their electricity. So I'm not sure what the final decision was on that.

Okay. Thank you.

We have a question from the live stream. How can this type of VR therapy be applied to individuals with autism?

INDIRA: That is interesting. It's not an area I've particularly looked at. But I know there's a lot of work going on with autism in virtual reality. I've just been to a symposium in Nashville this weekend, and there was someone talking about a therapy they've developed.

Because they worked with their child, who's autistic, and realized that through virtual reality, he started to play more, and could learn about how to respond to facial expressions. So there is work going on in this area. And it is one of sort of big areas of virtual reality.

Hi. I'm Aurelia. It seems like with this user group, you had to be very, very careful and do a lot of testing. I was interested in how you tested the script, and whether with this user group, did you have to test more often and more carefully, because they were... You know, with disabilities?

INDIRA: I wouldn't say we had to do it more often. It was just to try and get a really broad amount of people from around the country. So that's why we did it at multiple places. So we could see how people in Manchester thought and what people in Nottingham thought. So it was just that idea to get as many people with lived experience as input, so we could start to really see patterns and make sure that it wasn't just one person who thought this. This was a reason, and why those reasons were. With the script, it was interesting.

Because the first time I did a workshop on it, I basically just had a series of questions that I asked people to first of all write down on post-it notes, and they were questions like: If your friend was having a hard time, what words of encouragement would you use to help them? What words really do you not like if someone is trying to encourage you? So it was really basic questions around not necessarily even in a therapy, but just in general life, how would you want encouragement, and how would you encourage others? This then...

We did the same, when we went to... I think it was Bristol. We did those similar questions. But then I also created mock scripts. That people could then fill in. So at the end, they started to write their own scripts. Again, we then did a sort of partners' workshop, where we went through all of this, with the script writer, and worked it out. Then we took the script to the League to get their input of how they thought it was. The most important thing was: It wasn't patronizing. It made people feel it was their therapy, not that they're sort of being forced into things, and slightly false. Which again, can be hard when you've got a virtual therapist who's got a set sort of script to say. So it was really important to get that tone of voice right.

Cameron. You all met me already. The use case you're describing -- it seems like actually it's just kind of good for a lot of people. Especially anxiety. Right? So many people experience anxiety. And I'm wondering if the principles that you identified are universal principles. So, for example, or could be... More broad than just psychosis. Do you think that... Well, I'm curious what you think of that. And also, if there was anything that was specific to psychosis, that would not translate well to a general population.

INDIRA: I don't know if I would be fully able to answer that, not being a psychologist. But I know that a lot of it is around social anxiety. So it's interesting you saying about the elevator earlier. The therapy that we showed people, that they were developing, doing the study at the moment, included an elevator. It was for anxiety. And the ideas behind this, the psychological principles that psychologists use for therapy -- so one of them also was mirroring. The idea that someone does it ahead of you, and then you copy what they've done. So it used a lot of general psychological principles. But to be honest, I wouldn't want to answer fully, not being a psychologist.

Sure. I had a couple other questions. If I'm not... If I'm okay. So the methodology you came up with sounds very similar in a lot of ways to Lean methodologies and design sprints. Did you look at those as examples? Or would you consider using more formalized methodologies, from, like, user-centered design, or feedback, and that kind of thing? Lean design, et cetera?

INDIRA: Yeah, originally we had planned to use a double diamond approach. Which would have worked well. Probably because our time was cut short, we ended up with a more iterative approach. But I've got a development background. So I quite enjoyed that iterative approach, and found that having that constant... Because we had to go between the different groups, having that constant feedback loop, the slow design that built up over time worked quite well for this project. It might not have worked well for every project, but for me, it worked in that we got the most input from people with lived experience, and it was a way that we could bring everyone together as well.

For the non-technologists in the room, what's double diamond?

INDIRA: It's -- no, it's a user experience sort of... You've got a diamond up and down and then another one. So it's sort of... I should have got a graphic of it, actually.

Discover, define, and so you start off small in the discovery phase, and you just go out big, and you're like anything's possible, and then you make that into a definition of actually what is it we're going to do. And then you go out again. How are we going to do it? And you go out again, all big. And you go down into the last end of the diamond and it's going to go into... Which does work really well. And I suppose in some ways, because a lot of this had already been defined, because we knew it was gonna be a VR therapy, we knew it was going to be a coach, and the sort of basic structure, a lot of that definition phase had been done. So probably that's as well why double diamond didn't work in this case.

Hello again. I'm curious as to the feedback loop. I know it's currently being evaluated in England. I'm curious if you had any feedback loop mechanisms in place of how reporting looks like and what progress looks like in the end.

INDIRA: Yeah. So for the design, as I said, we had this criteria of usability. And engagement. So we got people... I actually haven't got them with me. A number of questions they had to answer when they tried it, and to get basically positive responses to those. And it was things like... If you haven't used it for a while, is it easy to come back to? If you make a mistake, is it easy to rectify? That you can work out quickly how you're going to do that.

So that was on the design side, we had a very definite -- almost like sort of usability questions. With the -- whether the therapy works or not, we won't know until the end of the trial. Yeah. We're hoping it will. There are previous trials. The ones I mentioned in the talk and there have been other ones have shown that this therapy can work for mental health therapies. But we'll see if this one does.

Questions?

Hi, sorry. Just one last question. When you had people with psychosis in your working groups, during the design and development phase, how many people were there? And how did you decide on the right number of people for it to be a representative enough sample?

INDIRA: They were people with lived experience. So most people weren't really ill still. So they were at the point of recovery. And better. So in total, I think 53 people with lived experience of psychosis were involved in the design of the project. When we ran the workshops, there was between 6 and 10 people. And in the weekly meetings, there was about 4. And in that, so we would have 4 people with lived experience, there would be me as a designer, and sometimes a colleague who was a designer. We would have one or two psychologists. The developers. And normally someone from the McPin Foundation came along as well.

Hi. My name is Breanne. I'm a UI/UX developer. Thank you so much for this. My question is: If the trials don't go well, by a certain standard, will you iterate on this? What will change, if that's the case?

INDIRA: Do you know, I'm not sure about that. We were funded for the three years, so this project runs... I don't know. If they came out and found there was something that didn't work, but it was a technical thing, I would think that could be fixed. If it didn't work as a treatment, I think that would be something different. And maybe they would look at a different approach. But I don't know what would happen to the treatment as a whole, if it didn't work.

Thank you.