Direct UIA Access to Web Content Processes

21 March 2025

Before settling on the project to implement a full accessibility cache in Firefox, I investigated several other alternatives. In my last post, I discussed asynchronous accessibility APIs. Another alternative I considered is switching to UI Automation and having the UIA tree accessed directly in web content processes, rather than communicating via the main UI process. While this is not possible with IAccessible2 because of browser sandboxes, it could theoretically be possible with UIA because it has an intermediary Windows component called UIAutomationCore which sits between the client and the server. As an operating system component, there are less concerns with regard to allowing it in the browser sandbox. Unfortunately, this proved to be infeasible then and I think it is still infeasible now:

Why UI Automation is Insufficient as an Accessibility API for the Web

19 March 2025

UI Automation (UIA) is Microsoft’s recommended accessibility framework for Windows, replacing the earlier Microsoft Active Accessibility (MSAA) framework. Despite this, to access web content, screen readers such as NVDA and JAWS continue to use IAccessible2, an open source API based on MSAA. This is not just because of the significant effort involved in switching to a different API, although that is certainly a factor. More importantly, it is because UIA is currently insufficient as an accessibility API for the web. Here are some of the reasons:

Making the Play/Pause Button on Earpods Skip to Next/Previous Track on Windows

3 February 2022

I recently got a new laptop: Dell XPS15 9510. While this is a pretty nice machine overall, its audio drivers are an abomination. Among other things, the Waves MaxxAudio software it ships with eventually leaks all of your system memory if you use audio constantly for hours, which is the case for screen reader users. I eventually got fed up and disabled the Waves crap, but this makes it impossible for me to use the headset mic on my Earpods.

Our Lounge Room Entertainment Setup

9 October 2012

Introduction

For a while now, we've had a Samsung LCD TV, Samsung Blueray player and Palsonic PVR in our lounge room, as well as an old 2005 notebook running Windows 7 connected to the TV for watching video files, media on the web, etc. I've recently made some major enhancements to this setup. I think they're pretty cool, cost effective and don't require lots of different devices, so I thought I'd document them here.

Responsibility for Windows Application Accessibility

3 March 2011

When an assistive technology (AT) user discovers an application that is inaccessible in some way, they will generally hold one of two parties responsible: the AT developer or the application developer.

In the Apple world, the application developer is generally responsible for ensuring accessibility. Users don't tend to complain to Apple when an application is inaccessible; they complain to the application developer. More often than not, this is correct. An accessibility framework has been provided to facilitate application accessibility and Apple's assistive technologies utilise this framework, so it's up to the application to fulfil its part of the bargain.

In contrast, in the Windows world, the AT developer is generally held responsible. In the past, before there were standard, fully functional accessibility frameworks, I guess this was fair to some extent because application developers had no way of making their applications accessible out-of-the-box. As a result, AT developers worked around these problems themselves through application specific scripting and hacks. However, Windows has had standard rich accessibility frameworks such as IAccessible2 and UI Automation for several years now. Therefore, this is no longer an acceptable justification. Despite this, the general expectation still seems to be that AT developers are primarily responsible. For example, we constantly receive bug reports stating that a certain application does not work with NVDA.

Some might argue another reason for this situation is that application developers have previously been unable to test the accessibility of their applications because of the high cost of commercial ATs. With the availability of free ATs such as NVDA for several years now, this too is no longer an acceptable excuse.

So why is this still the case in the Windows world? If it's simply a ghost from the past, we need to move on. Maybe it's due to a desire for competitive advantage among AT vendors, but the mission of improving accessibility and serving users as well as possible should be more important. If it's resultant to poor or incomplete support for standard accessibility frameworks, ATs need to resolve this. Inadequate or missing support for accessibility in GUI toolkits is probably part of the problem. We need to work to fix this. Perhaps it's because of a lack of documentation and common knowledge. In that case, the accessibility/AT industry needs to work to rectify this. Maybe there just needs to be more advocacy about application accessibility. Are there other reasons? I'd appreciate your thoughts.

Whatever the reasons, I believe it's important that this changes. Proprietary solutions implemented for individual ATs are often suboptimal. Even if this wasn't the case, implementing such solutions in multiple ATs seems redundant and wasteful. Finally, the more applications that are accessible using standard mechanisms, the more users will benefit.