At CSUN this year, I attended the open source math accessibility sprint co-hosted by the Shuttleworth Foundation and Benetech, where major players in the field gathered to discuss and hack on various aspects of open source math accessibility. My team, which also included Kathi Fletcher, Volker Sorge and Derek Riemer, tackled reading of mainstream math with open source tools.
Last year, NVDA introduced support for reading and interactive navigation of math content in web browsers and in Microsoft Word and PowerPoint. To facilitate this, NVDA uses MathPlayer 4 from Design Science. While MathPlayer is a great, free solution that is already helping many users, it is closed source, proprietary software, which severely limits its future potential. Thus, there is a great need for a fully open source alternative.
Some time ago, Volker Sorge implemented support for math in ChromeVox and later forked this into a separate project called Speech Rule Engine (SRE). There were two major pieces to our task:

1. SRE is a JavaScript library and NVDA is written in python, so we needed to create a "bridge" between NVDA and SRE. We did this by having NVDA run Node.js and writing code in Python and JavaScript which communicated via stdin and stdout.
2. One of the things that sets MathPlayer above other math accessibility solutions is its use of the more natural ClearSpeak speech style. In contrast, MathSpeak, the speech style used by SRE and others, was designed primarily for dictation and is not well suited to efficient understanding of math, at least without a great deal of training. So, we needed to implement ClearSpeak in SRE. Because this is a massive task that would take months to complete (and this was a one day hackathon!), we chose to implement just a few ClearSpeak rules, just enough to read the quadratic equation.

Our goal for the end of the day was to present NVDA and SRE reading and interactively navigating the quadratic equation in Microsoft Word using ClearSpeak, including one pause in speech specified by ClearSpeak. (ClearSpeak uses pauses to make reading easier and to naturally communicate information about the math expression.) I'm pleased to say we were successful! Obviously, this was very much a "proof of concept" implementation and there is a great deal of further work to be done, both in NVDA and SRE. Thanks to my team for their excellent work and to Benetech and the Shuttleworth Foundation for hosting the event and inviting me!
As a result of this work, I was subsequently nominated by Kathi Fletcher for a Shuttleworth Foundation Flash Grant. In short, this is a small grant I can put towards a project of my choice, with the only condition being to "live openly" and share it with the world. And I figured polishing NVDA's integration with SRE was a fitting project for this grant. So, in the coming months, I plan to release an NVDA add-on package which allows users to easily install and use this solution. Thanks to Kathi for nominating me and to the Shuttleworth Foundation for supporting this! Watch this space for more details.

Because NVDA is free software, we do not have the resources to provide free, direct technical support to users. Therefore, the NVDA-support email list was set up as "a place where users of the NVDA screen reader are able to ask questions about how to use NVDA".

A common complaint about mailing lists like this is that they produce a lot of messages. Some users cannot (or do not wish to) handle this high email traffic and therefore end up unsubscribing from the list fairly quickly, thus limiting its usefulness. Instead, users contact us directly or are driven away from the project. When directed to the tracker and email lists, one user who contacted me directly complained about having to "sign up to a thousand lists".

To combat this, we decided to selectively moderate the list, as full moderation is too time consuming. Users who broke (or bordered on breaking) list rules or otherwise had the potential to generate a lot of unnecessary traffic were moderated. Any post from those users that was irrelevant, unnecessary or might start such a thread was rejected.

Unfortunately, several users have been unhappy with or even outright offended by this. Today in particular, I rejected a post from a user (previously moderated for an off-topic post) which, while intended to be helpful, provided an incorrect (or at least very indirect) answer which I believed would cause more questions than it answered. No accusation was made, but this user took this very personally and made it clear that he would no longer support the project in any way.

Another common gripe is that users are often told to read the documentation when they ask questions. If it seems that a user hasn't even tried to read the documentation before asking a question, I do not think this is unwarranted. If they've at least tried and don't understand, this is a different matter entirely. If they don't wish to make the effort to at least try to understand the documentation, they should not expect free support.

It seems I can't win. I tried to do what I thought best for the NVDA community in limiting the traffic on the list so more users would be encouraged to use it. As a result, I'm accused of being unfair, draconian and ungrateful. Therefore, I've disabled all moderation on the list and I am withdrawing from the list myself for a while. I am done with support for now.

The following email sent to NV Access administration yesterday is an amazing mastery of politeness, eloquence, intellect and linguistic ability. I've reproduced it verbatim below, except for the obfuscation of some words for reasons that will become clear as you read. Enjoy!

From: Dave. I lost my cookie at the disco. <computerguy125@****>
To: admin@****
Date: Sun, 13 Feb 2011 23:40:16 -0500
Subject:

hEY MOTHER F****R YOU SCREWED UP MY LAPTOP.  fIX YOUR SCREEN READER.  bLIND ASS MOTHER F****R.  f***ING BLINKY.  cHANGE YOUR SHORTCUT KEY SO IT DOESN'T CONFLICT WITH SYSTEM ACCESS TOO.  yOU DON'T KNOW THAT THEN LOOK IT UP.  mOTEHR F****R.

-- 
Email services provided by the System Access Mobile Network.  Visit www.serotek.com to learn more about accessibility anywhere.

As Windows screen reader users will know, there is no screen reader included in Windows. Instead, users requiring a screen reader must obtain and install a third party product. Yes, there is Microsoft Narrator, but even Microsoft know that this is hardly worthy of the name "screen reader". :)

A few years ago, Apple revolutionised the accessibility industry by building a fully fledged screen reader, VoiceOver, right into Mac OS X. Ever since, many have asked why Microsoft can't do the same for Windows. Many are angry with Microsoft for this continued lack of built-in accessibility, some using it as support for the "why Apple is better than Microsoft" argument.

Here's some food for thought. I'm not sure Microsoft could do this even if they wanted to; their hands are probably tied in a legal sense. If they did, they could very likely be sued by assistive technology vendors for anti-competitive conduct, just as they have been sued several times concerning their bundling of Internet Explorer with Windows. Once again, Apple don't have to be concerned with this because there wasn't an existing screen reader on Mac OS X and they don't have the dominant position in the market.

I have no evidence for this argument. Perhaps I'm wrong, but history suggests that it is highly likely that I'm not.

Even as one of the lead developers of NVDA, I'm first and foremost a blind user who wants the best possible access, both for myself and other blind users. As such, I would very much welcome a screen reader built into Windows. Competition is good. A built-in screen reader doesn't mean that other screen readers can't exist. If the built-in solution were good enough, then there would be no need for NVDA to exist. If it weren't, NVDA would drive accessibility to improve through innovation and competition.

Since VoiceOver came to Mac OS X, many screen reader users have joyfully sung its praise - and there's certainly a lot to praise. Some who were previously Windows users have taken the plunge and moved to Mac OS X completely, never looking back. Others use both operating systems or perhaps dream of no longer being dependent on Windows for certain software. This is all fantastic, but as an NVDA developer, there are also aspects of this that frustrate and exasperate me.

Right from the start, NVDA has used "object navigation" in order to review the user interface, particularly to provide access to elements which can't be otherwise accessed using the keyboard. In short, every element in the user interface is generally represented by an object. These objects exist in a hierarchy or tree of objects. Object navigation allows the user to explore this hierarchy by moving between objects and then descending/entering/interacting with objects of interest. For example, entering a list would allow you to see its list items. This is in contrast to the "flat review" (or "screen review") method that Windows screen readers have traditionally used for review, whereby the user can review the content of the screen in a flat fashion from top to bottom, left to right, similar to the way a simple text document would be read. While this might seem more logical at first, it's worth remembering that a sighted user doesn't necessarily read the screen in this ordered fashion. Instead, they are more likely to focus on specific elements of interest, which in some ways is more akin to object navigation. As always, both methods have advantages and disadvantages.

On the Mac, there is only object navigation; there is no concept of flat review of the entire screen using the keyboard. (In the latest version of Mac OS X, you can explore the screen in this fashion using the trackpad. However, NVDA's tracking and reading of text under the mouse allows you to do something similar.) It seems that Mac users are quite happy with this approach, and yet we are constantly bombarded by NVDA users with complaints about the difficulty of object navigation and requests for flat review functionality in NVDA.

So, here's my question for those of you who are either full or partial Mac converts. Why is object navigation quite acceptable on the Mac, but yet not acceptable in Windows? It sometimes seems to me that some of the same users who frequently sing the praises of VoiceOver then turn and complain about the lack of flat review in NVDA. Perhaps I'm wrong and those users who want flat review would not be comfortable using VoiceOver. Even so, it sometimes seems that people are willing to accept a different approach on a completely new platform, yet are unwilling to accept it in a newer product on an existing platform. It's certainly true that NVDA's object navigation needs to be cleaned up a bit (removing extraneous objects, etc.), but I don't think this is the whole story.

ON the web, this gets even more interesting. Due to the non-linear fashion of web pages, Windows screen readers have had to provide their own "flat" representation of web pages. They then override the cursor and other keys to navigate within that representation. Aside from the technical issues associated with this, working with interactive controls on web forms requires a separate mode of interaction (named forms mode, focus mode, etc.) where the screen reader lets the user interact directly with the control by allowing cursor and other keys to pass straight through to the control. (Some screen readers can automatically switch to this mode when appropriate, but there are still two modes.) This is becoming more of a challenge with the ever growing number of web applications, where more keys are required to work with the application. Object navigation solves this problem because the cursor and other application keys are never overridden by the screen reader, so the screen reader doesn't interfere with the functionality of web applications.

Again, VoiceOver uses object navigation on the web and VoiceOver users appear to be quite happy with this. NVDA currently does what other Windows screen readers do, but again, would you be happy if we abandoned this approach in favour of object navigation? It would certainly solve this web application problem once and for all. I get the impression that NVDA users would be unhappy if we changed this.

I want to hear your thoughts. Comment here, Twitter or send me an email.