← All speakers

Joanne Song

Conference affiliation: The New York Times · 2026

Joanne Song is a mobile engineer exploring adaptive game accessibility: interfaces that respond when a player struggles to see, reach or navigate a control. Her experimental work with Shafik Quoraishee, presented while representing The New York Times at AI Engineer World’s Fair 2026, considers how on-device agents could adjust a game to the person playing it.

On this page

Joanne’s starting point is a limitation of conventional accessibility settings. An easy-mode toggle can change a game’s difficulty while leaving its controls inaccessible. A crossword might still have cells too small to tap reliably, an active-cell highlight that a low-vision player cannot see, or timing requirements that fail to accommodate someone’s processing speed. She grounds her approach in the perceivable, operable, understandable and robust principles of WCAG 2.2, then asks how software could respond to barriers as they occur.

Interfaces that can intervene

In her portion of the joint presentation on local mobile-game agents, Joanne develops two concrete interventions:

  • Keyboard-focus trap detection: An agent could monitor navigation through a dialog, recognize an endless focus loop and introduce an exit. For someone using a keyboard or switch instead of touch, that intervention could restore access to the rest of the game.
  • Dynamic touch-target resizing: An agent could inspect the interface and enlarge controls that are difficult to hit. Shaky taps could signal a need for larger targets or more forgiving input handling.

Joanne connects these interventions to adjustable input tolerance and task granularity: accepting less precise movements or breaking a complex sequence into smaller steps. Eye-gaze estimation could help detect difficulty finding something on screen, while handwriting recognition could offer another way to enter input. Her proposal treats accessibility and challenge as related adjustments that change with the player’s immediate needs.

What local personalization still requires

On-device personalization would keep sensitive interaction signals on the player’s device and avoid a cloud round trip for each adjustment. But an agent must improve access without making the game stutter. Joanne identifies planning within a roughly 16-millisecond frame at 60 Hz as a goal, alongside models that can predict the consequences of a layout change before applying it.

Long-term memory would let an agent learn one person’s habits and needs. A shared representation of game state could let it work across games without being rebuilt for each release. Joanne pairs those ambitions with a demand for better hardware and accessibility benchmarks that demonstrate whether the interventions actually help players.

The work remains experimental. At the presentation, Joanne and Shafik distinguished their research from published New York Times Games products, stating that the games had no AI features and that the puzzles were made by people.

Read the topics behind these talks

1 conference talk

Loading more talks and highlights…

References