▶ Watch ↗AI Engineer World's Fair 202618:04
Conference affiliation: The New York Times · 2026
Joanne Song is a mobile engineer exploring adaptive game accessibility: interfaces that respond when a player struggles to see, reach or navigate a control. Her experimental work with Shafik Quoraishee, presented while representing The New York Times at AI Engineer World’s Fair 2026, considers how on-device agents could adjust a game to the person playing it.
Joanne’s starting point is a limitation of conventional accessibility settings. An easy-mode toggle can change a game’s difficulty while leaving its controls inaccessible. A crossword might still have cells too small to tap reliably, an active-cell highlight that a low-vision player cannot see, or timing requirements that fail to accommodate someone’s processing speed. She grounds her approach in the perceivable, operable, understandable and robust principles of WCAG 2.2, then asks how software could respond to barriers as they occur.
In her portion of the joint presentation on local mobile-game agents, Joanne develops two concrete interventions:
Joanne connects these interventions to adjustable input tolerance and task granularity: accepting less precise movements or breaking a complex sequence into smaller steps. Eye-gaze estimation could help detect difficulty finding something on screen, while handwriting recognition could offer another way to enter input. Her proposal treats accessibility and challenge as related adjustments that change with the player’s immediate needs.
On-device personalization would keep sensitive interaction signals on the player’s device and avoid a cloud round trip for each adjustment. But an agent must improve access without making the game stutter. Joanne identifies planning within a roughly 16-millisecond frame at 60 Hz as a goal, alongside models that can predict the consequences of a layout change before applying it.
Long-term memory would let an agent learn one person’s habits and needs. A shared representation of game state could let it work across games without being rebuilt for each release. Joanne pairs those ambitions with a demand for better hardware and accessibility benchmarks that demonstrate whether the interventions actually help players.
The work remains experimental. At the presentation, Joanne and Shafik distinguished their research from published New York Times Games products, stating that the games had no AI features and that the puzzles were made by people.
▶ Watch ↗AI Engineer World's Fair 202618:04
Loading more talks and highlights…