Designing a Participatory Design Workshop | Accessibility VR Meetup Recap

INDIRA: Hi. Yeah. My name is Indira. And I work at the Helen Hamlin Center for Design at the Royal College of Art.

I also give talks about VR and Arduino. I want to talk today about designing a participatory design workshop. About two years ago, I started on a project called GameChange, which is -- oh, I think you can hear? Which is a virtual reality therapy for people with psychosis to help them feel more confident in everyday situations.

And a really important part of this was the participatory design co-designing, with people with lived experience of psychosis. So I want to talk a bit about how that worked today, and I also thought it might be quite nice, because it's in VR and we can move about the space, is to go through one of the workshops. So part of the co-design -- we ran workshops around the UK with people with lived experience of psychosis.

So I thought today it would be quite nice to just go some of the workshop, how I designed the workshop, how it was maybe different from other workshops I designed, and then do almost like a role play of that workshop. So just to go a bit about the project, this slide shows the people who were involved in the project. So we got funded by the National Institute of Health research. I think it was about 4 million pounds.

And the University of Oxford -- psychologists from there were heading this. So they've done work in virtual reality before. For different mental health conditions. And worked with people with psychosis.

So they were leading it. They created a spinoff company called OxfordVR to build the VR. And we worked also closely with a company -- a foundation called the McPin Foundation. They work with people with mental health issues, to make sure they're involved in the research into mental health.

So with this, they set up a lived experience advisory panel. And they're still meeting. Because this project, at the moment, is in its second year. It's just starting the third year.

So in the first year, it was the design and development of the project. And then the VR was actually built, and it's now in clinical trials for 132 people in England, to see if the virtual reality can help. I'll go more into what the virtual reality is, when I go into the bit that's the actual workshop. But to just sort of say: It's a standalone therapy, and people are in different environments that they can try out, and see if they can use different methods to control their anxiety.

So I don't know if people know about psychosis. I thought I would put a brief definition. It's a mental health problem that causes people to perceive or interpret things differently from those around them. This might involve hallucinations or delusions.

And psychosis often leads to social isolation. People feel anxious. And worried about going outside. For a number of reasons.

And they use -- this makes them socially isolated, which then can cause other health problems as well. So it can sort of be this sort of spiral. And I just wanted to show this slide. This is a really rough idea of how the project was set up.

So in the first year, where I was heavily involved with the design of the project, the design of the virtual reality and building the virtual reality, and as I say, OxfordVR built it, but we worked with people with lived experience of psychosis in the design, and workshops were one part of that. We also had design sessions where we worked with people with lived experience, plus the psychologists, and the developers, and the designers ourselves, and we met almost weekly over a number of months, and that was a chance for people to put ideas together, to quickly find out if things worked or not, get feedback between all the parties, so, for example, if we had one idea where people in the virtual reality would be catching balloons, but because we had the developers there, they could tell us that actually catching balloons is quite

a difficult thing to do in virtual reality. To make sure the physics is right. So we knew sort of straight away that wouldn't work. So it was really good having that sort of quick feedback.

We had the lived experience advisory panel that was set up by the McPin Foundation. They're still meeting. They meet a number of times each year. And in the first meeting, they helped us design these scenarios, the different environments that would be in the VR.

They said which ones were most important. And we could also go to them remotely. We can go to them remotely as well. And ask them questions.

And they also helped us set up the workshops. So it's actually the workshops that I wanted to talk about today. Was about how we went around designing the workshops. As I said, we had -- well, in the end, 7 workshops with people with lived experience of psychosis.

The reason these were slightly different from the other sessions was: These really were -- any idea was taken seriously. There wasn't any "nos". It's not like -- we can't do that in VR. Oh, no, that's not the psychology behind that.

This was people's thoughts and views and the way they wanted it designed. So we could really get -- people with psychosis, theirs was the main voice in these sessions. And at the start of designing, one of the first things we did was we talked to the psychologists, we talked to the McPin Foundation, we talked to the developers. Because we were all facilitators together in the workshop, and we did have quite a lot of facilitators in the workshop.

And we always made sure we had psychologists there, because some of the things that people might experience -- because everyone did try VR. They didn't have to, but everyone did try VR -- it could cause anxiety. So we always made sure we had psychologists there who worked with people who have psychosis. And we also made sure that there was space outside the main area that people could go to.

If they needed a break, as well as enough space in the room that they could bring support workers with them. Or they could bring family members, if they wanted to. So we wanted to make sure this was a really safe space that people could say what they want. If they didn't want to say anything, that was fine.

If they wanted -- we had enough facilitators that if they wanted to talk to a facilitator, they could talk for them or write something down for them, and express their views, that was available. So people could get involved. I don't know if you can still see this on the screen in VR, but this was -- I wanted to talk through today the second workshop we did. So in the second workshop, we had a series of aims.

And I suppose this is how anyone starts the design of any workshop. You work out what your aims are. And what it is you want to find out. And again, this was often done by talking to the psychologists and also talking to the developers, to see what they needed to be able to develop.

And it also -- we looked at previous workshops. So we always looked at previous workshops and feedback from the lead and the design sessions, to sort of decide what it is we need to know at this point. So this particular workshop -- we were looking at a number of things. There were some questions.

The therapy is inside VR. Everything takes place inside VR. So it is standalone and it's a virtual coach that walks you through and helps you decide which scenario you want to go into, and gives any advice and gives support. And we were looking at how that coach should look.

We had done some previous work on what people thought would make a good coach. And now we're looking at: If the coach had to be a free-standing person who was fully in the scene -- so full body, you could interact with that person, or if people would be happy with the person being inside a screen, so either a phone would come up and the coach would be there to give support, or an iPad or something like that.

We also were asked to check if people would mind having audio. So that was one of the things we looked at. The other thing we were looking at is: Though it didn't actually make it into the VR, but elements did, we were looking at having a space at the end that people could relax in. Somewhere they could go to sort of unwind before taking the headset off.

So we were looking at what people found relaxing. And this we did over a number of sessions. So that was one of the things we were looking at. And the last one was the overall flow.

So when someone comes into VR, how does it feel, where do they go, how do they choose a room to go to. What feels good in VR. What doesn't. So we sort of had those aims at the beginning.

And I don't know if you can see this, but this is just part of -- I write an agenda. So I just made an agenda of what we'd be doing, what the activity would be, what the materials that I needed to create would be. And I think most of the time I overcreated materials. I had a lot more materials than I needed.

And this meant there could be flexibility in the workshop. Because some things worked and some things didn't, and you couldn't always tell until you got there. You wouldn't necessarily know the space you were in until you got there. So I sort of overdesigned, in a way, and had lots of materials that I could use if needed.

So once that was all set up, we then went to the different places in England. And from this point -- this is actually the bit where I wanted to give you a taste of what the workshop was like. So the following slides are all from the workshop. So this is how we would come into the space.

We would all sit around in a group. We would all introduce ourselves. We would introduce the project. And we would normally then, at some point, either before or during, have lunch together.

We would sit and have lunch and chat. So that was that. I thought I would just give you an idea of what the workshop was. So I would start the workshop by introducing the Royal College of Art.

It's a postgraduate college of art and design. Where people do masters and PhDs. But we also do research, particularly in design. And the Helen Hamlin Center, where I work, is involved very much in people-centered design.

So we work with people who will be using the product. Whatever we're making. To help in the design process. And that's a very important part of it for us.

And that's why we're sort of running the workshop today, and we were brought into the project to work with people. So then I would just introduce the agenda. We would do an introduction, we would have an introduction to the project, into VR and each other. And then there would be lunch, followed by looking at the coach avatar, looking at the end of session, how the relaxation is, and looking at user engagement and how to keep people engaged and how the flow could help with that.

So let's see if I can see this a bit better. The screen. So the therapy itself -- as I said, it's a totally enclosed therapy. So it's standalone.

But there would always be a peer support worker with the person in it. And as it's self-contained, there's a virtual coach, as I said, who helps people through that. And then it will contain six scenarios. So these are virtual environments that are sort of everyday environments.

So they're a pub, a cafe, a street, a bus, a GP's waiting room, and a corner shop. And with each of these scenarios, there will be five different tasks that people have to do. And these tasks get increasingly harder. Not in that they're harder to do, but they're harder to stay in that situation.

So in the pub, for example, it starts off, and you're in the pub, and it's fairly quiet, and you're standing there, and you have to wait for someone to arrive. And then in the next level, you're doing the same thing, but this time there's more people in the pub. There's more noise. And then at one point, there's a football fan who will walk over to you and ask you to -- where the toilet is.

So you actually have to interact with someone that people might find causes anxiety. The sessions themselves, when people are in the therapy sessions, we imagine them to be about 30 minutes long. And the environments themselves -- these sort of images that we did actually share at the workshop -- because at that point, we didn't have the virtual environments built. They were still being designed.

But they're very much everyday environments. We wanted to make sure they're not London-centric. And we wanted to make sure they were a place that people recognized. So it's a general coffee shop.

It's a coffee shop like you would go in on a high street. The street, again, is a very generalized street. It's not old. It's not new.

It's sort of a general place where people live. It's not particularly -- it's not run down, but it's not really posh either. It's sort of that middle sort of environment. And the bus would be a single decker bus.

Partly because we didn't want the element of stairs coming in. Because people would have to go upstairs. So keep it on that level. A pub.

Again, a general pub that is on every high street. And with the GP's waiting room, there was a decision there that there would be glass in front of the receptionist. Just to make it feel -- it's a bit harder to deal with. It's a bit harder being in there.

And again, it was a corner shop. So it's just a shop that people go in every day. And the therapy is everyday situations. To help people evaluate the accuracy of their fears of being in that situation.

There are things called defense behaviors, that people have to get through these situations, if they have to go into them. So lots of people wouldn't go into the situation in the first place, but if they have to, they might only go at certain times of day, or they might rush through very quickly, to make sure they're out as quickly as possible, not looking at anyone, maybe wearing a hat or glasses. So this is to evaluate doing something differently, and seeing if that makes you feel less anxious.

And sort of see if the accuracy of the fears and by doing something -- if you can sort of lower those fears. And realize that outside the general world is a safe enough place to be in. And what we're looking at today is people keeping presence in VR. So once they're in that environment, they really feel that they're there.

How can we achieve that? The engagement, that people feel engaged in the process. And that it's easy to use. So that people feel that they can easily know what they need to do.

And the technology isn't hindering them. And so then it gives an idea of what the session is about. To develop ideas for the end session, to find out how people feel when they're presented in different ways. And how the flow of the application can be made better.

So those are workshop A. This is a slide from the workshop. So this is what people saw in the introduction. And then we got onto the workshop activities.

And that's something I would like to try today. I know we're not in a workshop and we haven't got Post-It Notes and things like that. But I have put around this room -- you can see a number of different things. So in this, everyone got the chance to try virtual reality.

And everyone actually did. So I've put over towards the balcony just a clip of some of the virtual reality they would have seen during this session. In this session, we didn't have any actual work from the developers, because they were still working on that. So we created -- a colleague of mine, Paul, created some prototypes.

So up in the balcony, I've just got some film that you can press play and you can watch, to get an idea of what people would have seen. And then over sort of opposite where the balcony is, there is a worksheet, which has the images of the coach, and for you to say strengths and weaknesses of a full length coach, of it being in a box, and voice only. And then going to the other side of that, there are some images of places to get ideas of whether people find those relaxing or not.

And then there's also the example of the worksheet that I created for that. And also the questionnaire that I created for that as well. To get an idea of what people find relaxing. So what I did, again, in this, it would have been post its in the actual workshop.

People put on the post its what they found to cause anxiety with those images. And what they found relaxing within those images. And then outside on the balcony is a flow diagram, which I think I can show through here. So yeah.

So there the two -- the coach pictures. The environment. And then there was a flow diagram. So this is a flow diagram of the actual application, where people come in.

What they see. What sort of branching there is at different points of it. And out on the balcony is a worksheet which looks at the ways that people might want to interact. What would make them want to stay in that space.

And today we're hoping -- you can go around and have ten minutes to look at these. Imagine you're in the workshop and you're doing these workshop activities. Because we don't have post its, we're hoping you would be able to draw on these. We seem to be having a problem with the pen.

I think maybe Thomas went through that. And so I don't know if you explained -- what we put instead is just go into the object. So the little wand, and find an object that you feel relates your feeling about that. So for the images, maybe go over and if there's an area that you find makes you feel anxious, put some sort of image that you feel represents that.

If you think there's something that's nice, something that represents that, and so -- because in the workshop, we did get people to split into groups and do things separately. So I thought it would be nice just to spend ten minutes to have a look around, have a look at the workshop materials, and try filling them in with the means we have in VR at the moment. So I don't know if Thomas, you want to add anything on how people can do that, or if you've explained.

THOMAS: Yeah. So if you all can hear me, this is our exciting experiment. I'm really excited that Indira wants to make this interactive. So unfortunately --

JOLY: I want hear you, Thomas. You're very muffled.

THOMAS: The way we should be able to mark things is not working.

Me too. I can't hear you.

THOMAS: If you use the wand icon, which is the pink icon, you can actually choose objects to insert.

INDIRA: Yeah. So if people want to use the wand icon, they can choose the objects. And if people want to go have a look around now, and then we'll join back here in a few minutes.

THOMAS: We'll help you. We'll take ten minutes and we'll help you.

INDIRA: And have a look around. And we can come back and I can sort of tell you what the reactions were to these, during the workshop, and what worked and what didn't. I'm gonna go over to the scene wall. If anyone wants to come over there and have a look at that.

I'm just gonna go and have a look at the video. So this is a prototype video that people would have been inside of VR. I'll just have a look outside on the balcony. In the balcony space, people could again put post its.

We had these up as A4 sheets on the wall. So the therapy room, start of session, the pause, end of session, and we asked people how they would enter the space, what would make them want to go back. To get an idea of how it could look.

THOMAS: So I put a couple objects over here. But this is all an experiment.

INDIRA: Hello?

THOMAS: Indira. Yeah. So I thought what was interesting for me... About this exercise -- if I was able to use avatars to communicate my feedback, I think in my coach avatar, I wanted to say (inaudible).

But if I had words or ways to communicate, other words, an object that I thought communicated that idea, (inaudible).

INDIRA: Going to the last wall.

THOMAS: Can you hear me?

INDIRA: Yeah, I can hear you.

THOMAS: Okay. I went back to my desktop. Because I stopped being able to hear on my --

INDIRA: Yeah, you're clearer now as well.

THOMAS: Okay. Yeah. So I just wanted to add that I love this idea. The writing wand should be working.

In Mozilla Hubs. Which we were thinking like -- to be able to tick or put something underneath the strengths or weaknesses. But I thought it was a cool idea to add --

I'm back out on the sea. Oh, no!

THOMAS: You're out on the sea?

Oh, hello! I didn't realize my mic was on.

INDIRA: It's a strange feeling when you end up over the water. It's quite interesting, though. I quite like it.

Right, right. I just thought everyone was standing there looking at me, and I was stuck out in the lake.

INDIRA: Oh, no. We didn't see you. We didn't notice you were there.

Nice to hear you, Indira. Sorry, Thomas.

THOMAS: Yeah, no. Thank you for being here. Like I said, this is all an experiment. So I was really happy that Indira wanted to do this.

Because we're trying to learn how to use these environments. And participate. So we want your feedback. And I already have feedback from what I was trying to do.

But I thought it was really interesting to be able to use this space. Like, we don't want to just have people coming and watching a presentation. We wanted to do interactivity. And we want to figure out how to do that.

Great stuff. Will do. I will comment for sure on the whole thing after, yeah.

THOMAS: Great. Does anyone want to contribute? Do you want to go back to your presentation?

INDIRA: I can go back to the presentation and people can always come and have a look at these after. So it was basically to say you can have a look and get an idea of what it was like to be in that workshop. Because then I wanted to quickly talk about what worked and what didn't. And what I learned.

In a way, we have the luxury of running 7 workshops. I learned a lot over that time of what worked and what didn't. This thing that is now out on the balcony didn't work. It's interesting I had actually planned to have this worksheet with it.

To get people to really think about what would be in each part of it. (inaudible) people were tired. And it's also -- I realized that I'd been thinking about flow a lot. You're designing your application.

You're thinking about where people come in. Where people branch off to a different environment. How they choose the environment. How they might pause it.

But it's something that's quite abstract, if you haven't been thinking about it for a while. So I think that didn't help. And at the end of the day, everyone was tired. So initially, I wanted people to have a brainstorm exactly like the one out on the balcony.

So they would see that and put ideas and work in pairs to come up with how that actually might work, and then fill out another worksheet. But in the end, though I say it didn't work, it didn't work how I planned, but people did come up with ideas on the balcony. They did put things on post its on walls. And some really good ideas came out of it.

Particularly around the pause and what it might be like. So the idea of the pause is if you're really anxious in VR, instead of taking the headset off and coming back to your actual environment, you still stay in virtual reality. And again, they weren't used in the therapy in the end. But it was just interesting to see some of the ideas that people came up with.

To do that, whether it's something that just freezes the environment, so you're still in the same environment, but everything stops, where the sound comes down slowly, or whether it goes to complete darkness, and you sort of get a complete break from what you're seeing. So while the workshop activity didn't work, some good ideas came out of it, and made me realize again how I needed to maybe develop the materials slightly differently.

But the other two worked quite well. So looking at the spaces that made people feel anxious or calm worked. So again, I thought about the pictures I'd chosen. And especially from the comments coming back, added different sort of -- made different pictures.

I wanted to make different pictures. And in other ways we talked about different environments. But this -- with people putting up post its of what they find relaxing or not, they gave really good feedback into things that I would find relaxing -- other people might find would make them feel vulnerable. Particularly if the space was too open, or there could be people hiding behind things.

And then this worksheet worked well, just to find out a general view on what relaxing environments would have in them. The colors. People getting to draw them. That worked really well.

And again, though we didn't have an end space that people could relax in, a lot of the work that we did on this did influence the coach's room. So when someone first comes into the environment, they go into a room that's different from all the other environments, where the coach can introduce them into the therapy and get them to choose a scenario. And at the end, they come back to that environment, so the coach can have sort of a -- how did that go?

How are you feeling now? Think about what you've learned through this. And that did, from these workshops, that did influence what that environment looked like. And again, with the coach, the coach was very interesting.

I'll go back to the coach. Because we went for a full bodied coach who is there, and while some people did like the idea of a coach in some sort of technology, in a phone or in an iPad-shaped thing, most people preferred that person was there with them. And so that's what we went with. So that was really useful to know.

And the lead went on to help with the general design of the coach in the other sessions. So really, my conclusions from running this workshop, and running a whole lot of them, was how important it is to take from one to the next. And as I said before, to keep almost -- to be able to keep it really flexible. So the design, while there were always very set aims, and I tried to have everything organized, I would definitely be very flexible on how it was delivered.

And if something wasn't working on the spot, come up with a different way of doing it. The one that's on the balcony, that flows -- when I realized that the idea of people pairing off didn't work -- we actually just had a general conversation of people who still felt like being in that conversation and discussing their ideas. So it became far more just an informal chat after the brainstorming. And throughout all the workshops, in this way, I would often change how an activity was run.

To make sure that people could get involved as much as they could. To have that safe space, where people could feel like they could leave at any time. Where they know who everyone in the room is, to make sure we've explained our roles properly and why we're there. And again, yes, generally just taking the feedback.

We always had a general feedback wall for VR and people could write notes on how they feel about VR. So just having different ways people could communicate with us. If they didn't want to speak, have the writing. If they didn't feel comfortable writing, have someone to talk to.

So people could get involved throughout. And that's the end of my presentation. I don't know if people have any questions. If you want more information on the project, that's the website.

And the Twitter handle. As I say, it's sort of starting the third year soon. The clinical trials -- soon we'll be getting close to the end. I'm not sure how the shutdown/lockdown has affected those.

But the aim is that if it works and gets through clinical trials, this would be used in the NHS. So the final bit of the work being done would look at how this could be used, the economy of this being used in the NHS, and how to get it it delivered in the NHS. Does anyone have any questions?

THOMAS: Yeah. I just wanted to say thank you, Indira, so much, for the presentation. And again, for working with us in this new environment. I guess my first question would be: Just from doing a user study, or some of the things we've seen in the room -- where did you see most of the Post-It Notes?

Or where did you see -- when you did this test in the real world, what did you -- what kind of feedback did you get for strengths and weaknesses?

INDIRA: Strengths and weaknesses. The strength of a full bodied coach is there is someone there with you, and you feel it's almost emulating somebody going out with you into the real world, and you feel that support. Whereas if it was called up on a phone, it would just feel a bit too remote, and it looked a bit funny. Someone was cut off.

The audio we actually knew before we asked that it wouldn't -- we guessed it wouldn't work. I wouldn't say we knew it, because we hadn't asked people. We guessed it wouldn't work, because one of the symptoms of psychosis is hearing voices. And having voices in your head.

So actually having a disembodied voice as a coach we felt wouldn't work. But we wanted to ask to see, to find out. And that was the general view. Yes.

It wouldn't work. There were some comments that people quite liked having a coach maybe in a mobile thing. They felt more control. They could bring that person out when they wanted to.

But the overwhelming response was they felt more natural to have a full bodied person there. So that actually then became -- so it really did influence the final outcome, so we did get a full body coach. And definitely, with the relaxation, it helped in finding that end space and designing the coach's room. I don't know if anyone else has any questions.

THOMAS: Does anyone out there have any questions?

INDIRA: Well, then, thank you for coming today. This is the first time I presented in VR -- started in VR and came to the laptop. It's been interesting to develop a talk for this. I love this project.

So it's always good to be able to talk about it. So thank you for coming today.