#### Update on AAM tests in WPT #### (fka Acacia) #### TPAC 2025 Note: # tpac 2025 acaia update --- ### What are the AAM tests again? Test [Core-AAM](https://w3c.github.io/core-aam/) and [HTML-AAM](https://w3c.github.io/html-aam/). --- #### Basic design for WPT AAM tests * Authored test are WPT [testharness.js tests](https://web-platform-tests.org/writing-tests/testharness.html), an HTML file that: * Contains markup under test: <br> **\<div id=test role=blockquote\>\</div\>** * Javascript calls the backend API and runs assertions. * Exactly like the GetComputedRole and GetComputedLabel tests. --- #### Basic design for WPT AAM tests * On the backend: * From python, query the accessibility API of the browser **just like an AT**. * Different "executors" for each platform will find the node in the platform's accessibility tree based on the **DOM ID** * Once you have the node, the executor can get information about that node. the execute statements related to that accessibility node. * Then it can send that information back to the front end. --- ### Status at last TPAC * Proof of concept branch on the Igalia fork of WPT * Test could be run locally on Linux, Windows and macOS * Discussed test design and some issues that will occur when landing in WPT * We will talk more about test design in a min! * Here is link to [last years presentation](https://notes.igalia.com/p/u3q-Rq8Cw#/). --- ### Things that happened since the last TPAC: * Not a lot of time 🙁 ... until some funding from NLNet! ☺️ * The funding is *only* for the Linux implementation. * Opened a [PR on web-platform-test](https://github.com/web-platform-tests/wpt/pull/53733) with successfully green CI and test runs on wpt.fyi! [Tests passing in CI](https://github.com/web-platform-tests/wpt/runs/48032728308), [wpt.fyi results](https://wpt.fyi/results/core-aam?sha=d508e4016d&label=pr_head). * Paused work until got thumbs up for wptrunner changes from wptrunner maintainers. --- #### Notes on the macOS and Windows interpretation * Contains "hello world" implementations. * Test is sent to the backend, and based on the DOM ID, it finds the accessibility node in the accessibility API. * If it can't find the node: Raises an exception * If it finds the node: Fails with string "test not implemented". --- #### Tests only run with: <br> `--enable-accessibility-api` * **./wpt run --enable-accessibility-api chrome core-aam/role/blockquote.tentative.html** * Flag does two things: * Installs relevant python dependencies * Turns on accessibility in the browser when the browser is started by wptrunner * This flag defaults to `false`, which means: * When PR lands, wpt.fyi will show failing tests because wpt.fyi does not run with `--enable-accessibility-api` * Goal is for `--enable-accessibility-api` will default to true. --- Long term goal: accessibility tests run in full runs of WPT, you don't need to specify an extra flag. **So, what are the prerequisites for switch <br> `--enable-accessibility-api` <br> to default to `true`?** --- **1. When we have a way to turn on accessibility only for a subset of tests.** * Current plan is to leverage wpt.fyi "chunking", which is the parallelization method * If a "chunk" has an accessibility "folder", then turn on accessibility for the browser that runs that chunk of tests. * Will work on this after the initial PR lands because it will require potentially significant changes to wptrunner, and I don't know the level of support those changes will have. --- **2. When the tests are not longer marked as `.tentative`** * Once they are not marked as `.tentative` they will matter more to consumers of WPT (success rate metrics). * Design of the tests is still up for debate. * The names of the test shouldn't very at this point, useful to be able to see history of tests over time. * The linux python bindings have a system dependency and wptrunner will fail during the virtualenv set up if that dependency is not met: don't want to break downstream consumers of WPT until test really ready. --- #### Goals for the [PR on WPT](https://github.com/web-platform-tests/wpt/pull/53733) * Current PR includes: * Initial necessary changes to wptrunner. * Tentative tests to show how the wptrunner changes work. * I'd really like to land this PR so that follow up PR are either: * Only changes to `wptrunner` so I only need WPT core maintainer review/feedback. * Only about test design, so I only need ARIA WG review/feedback. * To allow for this seperation: * This week: resolve a lot of the comments in the PR and get +1 from ARIA WG? * Next week: resolve feedback from wptrunners, land???? --- #### Goal for this meeting: What is the shape of the API the javascript calls? * **Return bag of properties** * Ruled out, too many properties and too many API end points that take parameters. * **Declarative test format** * [What's in the current PR](https://github.com/web-platform-tests/wpt/blob/5444fc6a4b6d89e318af57a503c86167b88599ac/core-aam/role/blockquote.tentative.html), based on the old manual tests. * Sends a list of test statements to the backend to verify. * Somewhat abstracts the API, the test statements look like Core-AAM bullets. * **Imperative test format** * In the javascript of the test, you include a string of python to be execute on the backend that directly queries the accessibility API. * Maximum flexibility in tests authoring ---