# Acacia ### Automated AAM Test Suite in WPT Val + Alice --- #### Goal: Extend WPT infrastructure to allow automated testing for AAM specifications - We want to be able to test platform API mappings directly, as well as testing the internal accessibility tree like `computedRole` and `computedLabel` ---  Note: Existing code allows testing internal computed role and computed name. This would add the ability to test that accessibility API mappings are correctly implemented. --- **🌳 DOM Element 🌳** to **🌳 Accessibility API Node 🌳** is standardized by the ARIA working group of the W3C in 📜 [Core-AAM](https://w3c.github.io/core-aam/) and [HTML-AAM](https://w3c.github.io/html-aam/) 📜 --- ### Platform Accessibility APIs: | OS | API | Browsers | | ----------- | ----------- | --- | | macOS | [AX API]() | Firefox, Chrome, Safari | | Linux | [ATK](https://gnome.pages.gitlab.gnome.org/at-spi2-core/atk/)/[AT-SPI](https://docs.gtk.org/atspi2/) | Firefox, Chrome | | Windows | [MSAA](https://learn.microsoft.com/en-us/windows/win32/winauto/microsoft-active-accessibility) with [IA2](https://wiki.linuxfoundation.org/accessibility/iaccessible2/start) | Firefox, Chrome | | Windows | [UIA](https://learn.microsoft.com/en-us/windows/win32/winauto/ui-automation-specification)| Chrome (in progress) | --- ### [We have a working prototype](https://github.com/Igalia/wpt/pull/2/files) * Extends `testdriver` infrastructure * This will allow us to use WebDriver commands (like clicks) intermixed with accessibility API queries * You can run the tests by following the instructions in the PR * Can be run on Linux, macOS and Windows, testing the relevant APIs for each OS --- ### And we have an [Web Platform Test RFC](https://github.com/web-platform-tests/rfcs/pull/204) * Any questions about the project or RCF? * We have questions for you in the next slides! --- ### Question 1: How to get the browser process? In order to query the browser process via the accessibility APIs, the easiest option is to find it via process ID. This means we need a way to get the pid for the browser under test. Options we've considered for this so far: a. Standardize returning processID in webdriver spec: [spec PR](https://github.com/w3c/webdriver/pull/1833/files) b. Use extension capabilities "moz:processID" (already exists), and "goog:processID" ([CL](https://chromium-review.googlesource.com/c/chromium/src/+/5825307)) and "safari:processID" (DNE) --- #### Question 2: wpt.fyi and python dependencies - In order to run these tests, for each API, we will need to install the API's python bindings. - How to introduce optional python requirements (requirements only for specific folders)? - Option 1: add them to `tools/wptrunner/requirements.txt`? - Option 2: Add [conditional requirements](https://github.com/web-platform-tests/rfcs/blob/master/rfcs/conditional_requirements.md) for `wpt run` based on flag (such as `--install-a11y-reqs`) - Option 3: ?? - Who manages the data collection for wpt.fyi? --- #### Question 3: Per Platform tests/subtests - These tests are inherently platform-specific. - Each OS may also require testing a different number of assertions. - e.g. `<input type="password">` requires one role and 3 states to be set in MSAA/IAccessible2, but a role, subrole and role description in AX API. - So, we would prefer a way to have platform-specific subtests or platform-specific tests, such that only the appropriate (sub)test runs and passes/fails on each platform ---  [Link to example test Results](https://spectranaut.github.io/examples/wpt/role_blockquote_test.html) --- #### Question 3: Per Platform tests/subtests - We don't necessarily need to fix this right away, since having redundant "Pass" results isn't terribly harmful - We'd like to eventually have a way to show an "N/A" for subtests, or simply not run/show any results at all for irrelevant tests/subtests --- #### Question 4: Ensuring a separate browser instance for Acacia tests - Acacia tests will need to turn on accessibility code in the browser, which can significantly impact performance - It's either not possible or not straightforward to disable that code again - So, we'd like to make sure that if browser instances are shared between WPT tests, that Acacia tests don't share browser instances with non-Acacia tests which may be performance-sensitive --- #### Question 5: Can we get a design review? - We've done our best with no previous knowledge of the codebase! - We'd really appreciate someone more familiar with WPT infrastructure helping us figure out the best way to integrate the relevant callbacks, if possible. - The [PR for our proof-of-concept](https://github.com/Igalia/wpt/pull/2) is linked in the [RFC](https://github.com/web-platform-tests/rfcs/pull/204)