Prior to releasing it, Apple promised iOS 14 would come with enhanced accessibility features. Sure enough, the company came through. Out of all the added accessibility enhancements, I want to share my experience with the new accessibility feature called VoiceOver Recognition, particularly Screen Recognition.
Although VoiceOver Recognition has been available for a few versions, Screen Recognition didn’t appear until iOS 14. The bad news is that this feature is only available for iPhone XS or XR and later. The good news is that I have an iPhone 11, so I had access to the new feature.
What Is VoiceOver Recognition?
Using on-device intelligence, VoiceOver Recognition helps the iPhone automatically improve the accessibility of apps, images, and texts. It has three different yet connected components: Image Descriptions, Speech Recognition, and Text Recognition.
To access them, go to Settings > Accessibility > VoiceOver > VoiceOver Recognition.
The Image Descriptions feature speaks descriptions of images shown in the app and on the browser. Screen Recognition automagically makes apps more accessible by recognizing items on the screen. And finally, Text Recognition speaks descriptions of text found in images.
Image Description and Text Recognition have been around for a while. What caught my interest is Screen Recognition. Some previously inaccessible elements such as buttons, sliders, and icons are now recognized by VoiceOver.
For instance, if you have a screen that does not announce anything at all or doesn’t recognize the controls present, the screen recognition feature will try its best to tell the blind customer what it does and 80% of the time, it is accurate.
Taking VoiceOver Recognition for a Spin
I started testing VoiceOver Recognition with the apps that I often use. First up is the Facebook app. Right away, I noticed a significant difference in how the phone describes the photos. It provides more details making it easier for me to visualize what’s in the photo. I tried it on Instagram and had the same positive experience.
Next, I put it to the test with the photos on my phone. The improvements made me happy. VoiceOver Recognition described all of my photos, screenshots, and captured text messages. Now I better understand what those images contain.
But what I wanted to know is how this would affect my experience with my mobile banking apps. I’ve always had problems navigating my bank account in these apps. Before this update, checking my balance was the only accessible task. Important functions such as transferring funds and paying bills hadn’t been accessible.
I started with the Bank of the Philippine Islands’ mobile banking app. Screen Recognition came in handy here. Using the VoiceOver rotor, I selected Screen Recognition and turned it on for this app. For those not familiar with the VoiceOver rotor, it allows you to change the VoiceOver volume or speaking rate, move from one item to the next like Tab, and other features.
Turning on Screen Recognition made a big difference. I can now find the buttons for transferring funds and paying bills. However, to properly access the list of available banks to transfer funds, I had to temporarily turn off Screen Recognition. Doing this helps VoiceOver function better. So, it’s a matter of figuring out when to turn Screen Recognition on and off to complete a process.
I switched to the Union Bank mobile app. My experience was almost the same as the app from BPI. There are parts where having Screen Recognition helps. And there are also times when it needs to be turned off to allow me to regain access.
It sounds a little tedious, but I didn’t mind. Though the updates added extra steps of turning the feature on and off, the benefits outweighed the drawbacks. The best part is that I could do these tasks independently.
Exploring the App Store
Another area I wanted to explore was Apple’s App Store. Before the update, it was a struggle to download an app because the image and descriptions of the app weren’t accessible. This sometimes left me wondering whether the app that I downloaded was the same app that I wanted.
Thanks to VoiceOver Recognition, it announces the apps and their associated content. It’s much easier to identify the different apps inside the App Store especially those with similar names.
While this is a huge step forward, it still has a few kinks. Since this feature is new, I know not to expect it to be working perfectly. While VoiceOver worked in the previously mentioned instances, it didn’t work in my emails and attached files with text images.
A good example is the emails I get from the Global Wonks platform. They send all announcements and ads as email images. The actual contents have text, but they appear as images. In all the emails they sent, I never get to read them unless I ask for sighted help.
I turned on screen recognition and opened their latest email. I could read some of the text. It’s definitely an improvement over not being able to read the content at all. Still, I couldn’t read all of the text.
Next, I opened an email from my primary bank. It contains my monthly bank statement attached as a PDF file. It’s been a problem for a long time and it still is. I tried to open the interface and got as far as entering my password.
Beyond that, even turning on Screen Recognition, I couldn’t bypass the password screen. I’m not sure if I managed to open the PDF file. If I did, I couldn’t review the contents of the document.
Final Thoughts on VoiceOver Recognition
Apart from the fact it only works on newer iPhone models and iOS versions, VoiceOver Recognition also has the tendency to render currently accessible features inaccessible.
As previously described, there are times when I needed to turn it off. That’s because some of the app’s features become inaccessible when I turned on VoiceOver Recognition. The same thing happened while browsing Facebook. The image and text recognition are adequate. But turning on Screen Recognition caused Facebook navigation to become inaccessible and confusing.
Yes, the new feature still has glitches and limitations. Knowing Apple, I believe the company will continue improving accessibility and the user experience. Overall, I’m happy with it because it delivered many improvements in the user experience of a blind iOS customer.
More Reads
If you liked this article, you may be interested in these:
- Slack Vs. Discord: Which Is Better for Screen Readers?
- Using Mozilla Hubs with a Screen Reader
- Seeing the World, Together with Be My Eyes
Sounds like VO recognition is great for the end user but for those evaluating apps for accessibility the feature should be turned off so that issues can be found and the developer can fix it properly.