Acacia (for ARIA WG)

Goal: Extend WPT infrastructure to allow automated testing for AAM specifications

• We want to be able to test platform API mappings directly, as well as testing the internal accessibility tree like computedRole and computedLabel

Check out the working prototype!

• Extends testdriver infrastructure
  • This will allow us to use WebDriver commands (like clicks) intermixed with accessibility API queries
• You can run the tests by following the instructions in the PR

Prototype works for linux, mac and windows

Accesses API using python bindings, see the code:

• Linux, Atspi
• Mac, AX API
• Windows, only MSAA + IA2

• This is a big enough change to land in WPT that it requires an RFC
• There are a few open technical questions, to be discussed later in the week:
  • how to get browser process ID (for querying the API)
  • how to ensure browsers are running with accessibility enabled for these tests (cf. chrome’s --force-renderer-accessibility flag)
  • how to query the accessibility tree at the right moment

Test design

What we most want your input on today is how you’d like to author tests, since we are still designing this system

Test design assumptions:

• One test file per “entry” in an AAM e.g. blockquote in core-aam
• Currently constrained by wpt.fyi having no concept of platform-specific tests or “not applicable” tests
• Modelled on existing testdriver-based tests, with the test logic and the HTML in the same file

Constraints of wpt.fyi until we have “not applicable”:

• Results will show up as “pass” on platforms other than the one the test is being run on, even though no assertions are run
• Example test results for a test run on Linux, which is only applicable for AT-SPI

Test designs we are considering:

1. “bag of properties” sent from the backend
2. “testable statements” sent to the backend

Bag of properties

link to test example

<p id="test" role="blockquote">quotable thing</p>
<script>
promise_test(async t => {
  const node = await test_driver.get_platform_accessibility_node('test');

  if (node.API == 'atspi') {
    assert_equals(node.role, 'block quote', 'Atspi role');
  }
}, 'Atspi: role blockquote');
</script>

Testable statements

link to test example

<div id="test" role="blockquote">quotable thing</div>
<script>
AAMUtils.verifyAPI(
  'role=blockquote',
  'test',
  {
    "Atspi" : [
       [ "property", "role", "is", "block quote" ]
    ],
    "AXAPI" : [
       [ "property", "AXRole", "is", "AXGroup" ],
       [ "property", "AXSubrole", "is", "<nil>" ]
    ],
    "IAccessible2" : [...],
    "UIA" : [...]
  }
);
</script>

Discussion

• How would you want to write tests? Any ideas beyond the two listed here?
• (If we have time) AT API experts: any advice on how to make sure we’re querying the accessibility tree only once it has actually been built?