diff --git a/rfcs/assets/accessibility_apis.png b/rfcs/assets/accessibility_apis.png new file mode 100644 index 00000000..d391cf16 Binary files /dev/null and b/rfcs/assets/accessibility_apis.png differ diff --git a/rfcs/assets/platform_accessibility_api_testing.png b/rfcs/assets/platform_accessibility_api_testing.png new file mode 100644 index 00000000..f7470bdd Binary files /dev/null and b/rfcs/assets/platform_accessibility_api_testing.png differ diff --git a/rfcs/assets/webdriver_accessibility_testing.png b/rfcs/assets/webdriver_accessibility_testing.png new file mode 100644 index 00000000..ea16d636 Binary files /dev/null and b/rfcs/assets/webdriver_accessibility_testing.png differ diff --git a/rfcs/wpt-for-aams.md b/rfcs/wpt-for-aams.md new file mode 100644 index 00000000..b664c088 --- /dev/null +++ b/rfcs/wpt-for-aams.md @@ -0,0 +1,473 @@ +# RFC 204: WPT testing for AAMs + +## Summary + +Extend WPT with the ability to test Accessibility APIs exposed by browsers. + +The Accessibility API Mapping (AAM) specifications +([Core-AAM](https://www.w3.org/TR/core-aam-1.2/), +[HTML-AAM](https://www.w3.org/TR/html-aam-1.0/), +[SVG-AAM](https://www.w3.org/TR/svg-aam-1.0/), potentially +[MathML-AAM](https://w3c.github.io/mathml-aam/), +[CSS-AAM](https://w3c.github.io/css-aam/)) describe how user agents +should expose semantics of web content languages to accessibility +APIs. These documents, taken together, describe how the browser should +build an accessibility tree for each web page, and how the tree should +be mapped to various accessibility platform APIs. + +## Details + +
+

Background: Computed Accessibility Tree, Platform Accessibility APIs and the AAMs

 + +#### The Accessibility Tree and Accessibility APIs +![Diagram showing the interaction between a browser and accessibility APIs. See text below for more detail.](assets/accessibility_apis.png) + +
+Full image description +A block diagram. + +At the top level it shows an assistive technology application, such as +a screen reader using speech or braille, communicating via Platform +Accessibility APIs with a Browser application. + +Within the Browser application, it shows that the Assistive Technology +is communicating via a Platform API mapping which is an adapter for +the computed accessibility tree data structure. In turn, the computed +accessibility tree is built based on other data structures: the DOM +tree and the layout/box tree; the layout/box tree is based on the DOM +tree and CSS. + +The computed accessibility tree falls in the center of the diagram, as +it's the source of truth for the platform API mappings which are +queried by assistive technology, and also the result of a complex +process of transformation based on several other data structures. + +At the bottom of the diagram is a legend explaining how to read the +different types of arrows and boxes used in the diagram. +
+ +Assistive technologies (ATs) query the browser via platform +accessibility APIs. These are operating system-specific APIs which +provide detailed information about an application's UI state for the +purpose of augmenting the user experience for users with disabilities +which affect their ability to perceive or operate the standard UI. + +One example of a commonly used AT is a screen reader, which assists +users with vision impairments by providing a spoken version of the UI. +Typically, a screen reader user will navigate through the UI by +traversing the page and "visiting" UI elements, hearing a spoken +description of each such as "Overview, heading level 2" or "Core-AAM, +link". + +The accessibility APIs allow this by making extensive data available +about the UI, in the form of a tree structure referred to as the +"accessibility tree". The accessibility tree provides information +about each UI element (often corresponding to a DOM node), allowing +screen readers to provide these spoken descriptions. Each platform's +accessibility APIs provide methods to query the tree structure, and +the information available for each node. + +They also typically provide ways to interact with the UI, such as +sending a "click" action to the currently visited UI element, or +selecting a particular option from a picker. Also, they provide ways +for the UI to provide real-time updates or alerts in case of UI +changing in a way that should be brought to the user's attention. + +#### The computed accessibility tree + +Since browser engines are typically platform-independent, typically they will compute a generic accessibility tree which can be adapted to any platform's accessibility APIs. +The platform APIs are typically supported using an adapter which maps the platform API on to data from the computed accessibility tree. + +#### AAM specifications + +The AAM specifications provide mappings between web technologies such as ARIA and HTML, and platform accessibility APIs. + +For example, here is the mapping from Core-AAM for an Element with [`aria-checked=true`](https://w3c.github.io/core-aam/#ariaCheckedTrue) + + + + + + + + + + + + + + + + + + + + +
MSAA + IAccessible2 + State: STATE_SYSTEM_CHECKED
+ Object Attribute: checkable:true +
UIA + Property: Toggle.ToggleState: On (1)
+ Property: SelectionItem.IsSelected: True for radio and menuitemradio +
ATK/AT-SPI + State: STATE_CHECKABLE
+ State: STATE_CHECKED +
AX API + Property: AXValue: 1
+ Property: AXMenuItemMarkChar: ✓ for menuitemcheckbox and menuitemradio +
+ +And here is the mapping from HTML-AAM for a [`
+ +
+

Prior art: Get Computed Label, Get Computed Role, potential WebDriver extensions

 + +![Diagram showing how WebDriver supports getting the Computed Label and Computed Role for an element. Full description below.](assets/webdriver_accessibility_testing.png) + +
+Full image description +A block diagram. + +This diagram extends the diagram in the +"accessibility tree and accessibility APIs" section, showing how +the WPT application can query the browser's Webdriver API +implementation to request the computed role and computed label for an +element. + +It adds a block for the WPT application to the left of the Browser application, +below the assistive technology application block, +and a block within the Browser application for the WebDriver API implementation. + +Where the assistive technology application queries the Platform API Mapping +via platform accessibility APIs, +the WPT application queries the WebDriver API implementation via HTTP. + +Like the Platform API mapping adapter, +the WebDriver API implementation also queries the computed accessibility tree. +
+ +WebDriver and testharness.js support getting the computed +accessibility +[label](https://www.w3.org/TR/webdriver2/#get-computed-label) and +[role](https://www.w3.org/TR/webdriver2/#get-computed-role) for a +particular element. This allows testing the two core properties for +nodes in the computed, platform-independent accessibility tree, +verifying that the first step of the process for supporting the +platform accessibility APIs has been successfully completed for those +properties. + +Computing the platform-independent role is supported in the AAMs, +particularly in HTML-AAM, by the addition to each mapping table of a +"computed" row in addition to the platform-specific API rows. In the +`
+ +### Proposal: testing AAMs via platform accessibility APIs directly + +![Diagram showing the proposed addition: querying the browser from WPT via the platform accessibility APIs. Full description below.](assets/platform_accessibility_api_testing.png) + +
+Full image description +A block diagram. + +This diagram extends the diagram in the +"Prior art" section. + +It adds a dotted line arrow from the WPT application to the Platform +API mapping adapter within the Browser application, indicating the +addition proposed here to use the Platform Accessibility APIs directly +to test the browser's support for those APIs. + +Notably, while the existing mechanism for WPT to query the browser via +WebDriver communicates with the WebDriver API implementation within +the browser via HTTP, the proposed addition bypasses WebDriver to +query the Platform API Mapping directly via Platform Accessibility +APIs. This matches the way that Assistive Technology communicates +with the browser application.
+ +As described in the "Prior Art" section above, we think testing the +computed accessibility tree is extremely useful in itself, and a good +fit for the platform-independent nature of browser testing and +WebDriver specifically. It provides a good foundation for ensuring +that the first stage of accessibility API mapping for web features has +been implemented, since in almost every case browsers will need to +implement the cross-platform tree as a prerequisite for implementing +the platform-specific APIs. + +This proposal focuses on an equally important goal, directly testing +that each browser's platform API support conforms to the AAM +specifications. The easiest and most reliable way to test that is to +use the APIs directly, matching the way that assistive technology +communicates with the browser. + +An alternative would be to extend WebDriver to mirror each platform API's vocabulary. +This would involve a great deal of work to specify how those vocabularies should be expressed +through WebDriver and testdriver, +and implementation work to implement those APIs and test the implementation, +before we could even write any tests using the WebDriver version. + +
+

Example: computed role vs. platform API mapping for <input type="password"> +

+ + +The table below is excerpted from the HTML-AAM mapping for `input` (`type` attribute in the Password state): + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
[wai-aria-1.2]No corresponding role
Computed Role
html-input-password
+ MSAA + IAccessible2 + +
Role: ROLE_SYSTEM_TEXT
+
States: STATE_SYSTEM_PROTECTED; IA2_STATE_SINGLE_LINE; STATE_SYSTEM_READONLY if readonly, otherwise IA2_STATE_EDITABLE
+
UIA +
Control Type: Edit
+
Localized Control Type: "password"
+
Other properties: Set isPassword to true
+
ATK +
Role: ATK_ROLE_PASSWORD_TEXT
+
States: ATK_STATE_SINGLE_LINE; ATK_STATE_READ_ONLY if readonly, otherwise ATK_STATE_EDITABLE
+
AX +
AXRole: AXTextField
+
AXSubrole: AXSecureTextField
+
AXRoleDescription: "secure text field"
+
+ +Since each platform accessibility API has its own vocabulary and its own conceptual framework, +the concept of "password field" is expressed differently in each API. + +Where the computed accessibility tree on each platform would contain a node +with the role `"html-input-password"`, +the browser would need to adapt that computed role to each platform API correctly +in order for users of the assistive technologies which query those APIs +to be able to use the browser. +
+ +# Test Design + +We have an [PR open on WPT for a new test +type `aamtest`](https://github.com/web-platform-tests/wpt/pull/53733) which is +modeled after `wdspec` tests and re-uses some of the same classes and +python test fixture. + +## Test files + +Like the `wdspec` tests, a test file is a python file. Each file +represents a mapping table in one of the AAMs including: (1) some html +markup, and (2) a subtest for each accessibility API. The WPT +webdriver client implementation is used to load HTML into the +browser. The accessibility APIs are provide via pytest fixtures. On +platforms where the subtest does not apply, the "PRECONDITION_FAILED" +test result is used. + +## Ensuring accessibility is enabled in the browser + +The introduction of the new test type allows us to restart the browser +when the wptrunner enounters the `aamtest`, or a test other than the +`aamtest`. When the next test is an `aamtest`, the browser is started +with accessibility enabled. When it leaves `aamtests`, the browser is +restarted without accessibility enabled. + +Firefox and Chrome both allow for the accessibility to be enabled from +a CLI argument. Safari does not yet, but there is a [PR to add this +functionality in the +works](https://github.com/WebKit/WebKit/pull/56965). Until this PR +lands, Safari will not support this test type. + +## Finding the browser through the accessibility API + +In order to query the browser application via accessibility APIs, we +need a reliable way to find the correct browser application. We can do +this by the application name (which means only one browser instance +can run at a time) or by PID. To run the test by PID, we need to get +the PID of the browser. + +Geckodriver exposes the PID in the capabilities as `moz:processID` and +Chromedriver exposes it as `goog:processID`. Safaridriver does not +expose the ID, until they do, testing Safari cannot be +parallelized (as only one safari instance can run in order to find it +by name in the accessibility API). + +Valerie has filed [an issue](https://github.com/w3c/webdriver/issues/1823) +on WebDriver to propose adding a mechanism to request the browser PID, +potentially through the `capabilities` object used when creating a Session. + +## Risks + +These test might be slow to to run. The browser is noticeably slower +when accessibility features are turn on (when the browser builds the +accessibility tree). Additionally, the browser can't run in headless +mode. + +The tests may be flaky due to timing issues with the accessibility +tree being built. The accessibility tree lags behind the DOM and even +paint, with no feedback available to the page about what the status of +the accessibility tree is. For now, this is handled by the use of +polling, with a timeout set to half of timeout for test otherwise. The +webdriver computed name/computed role APIs sidestep this issue, since +they build the accessibility tree on demand. + +These test introduce new python dependencies, and one of those python +dependencies (the linux API) replies on a system level package, and +installation of that package cannot be handled by the test suite. + +These tests introduce platform specific tests and results, which +increase the complexity of WPT. +