▶ Watch ↗31:22
Conference affiliation: Cohere
Nouha Dziri is a senior research scientist at Cohere Labs, working with Joelle Pineau on multi-agent language-model systems. Her research examines how models acquire reasoning skills, where those skills fail to generalize, and how training data and system design shape their behavior. Her work spans grounded dialogue, algorithmic reasoning, open-model safety tools, and coding-agent teams whose performance depends on how they divide the work.
Nouha completed her PhD at the University of Alberta and Amii in 2023, after joining Ai2 as a postdoctoral researcher in November 2022. Her earlier dialogue research examined a problem that begins before a model generates an answer: the supposedly correct responses used to train it can themselves contain invented information.
In a 2022 study of knowledge-grounded dialogue, Nouha and her collaborators found that more than 60% of responses in the benchmarks they examined contained hallucinations. Models trained on those conversations could amplify the problem. Improving factual grounding therefore required examining the training answers as well as the model’s output.
FaithDial, which Nouha co-authored, addressed that problem by editing unsupported responses in the Wizard of Wikipedia dataset into conversations grounded in the supplied knowledge. The resulting resource supported both dialogue generation and a critic that distinguished faithful answers from hallucinations. Human evaluations found that responses from models trained on FaithDial remained engaging and cooperative: grounding an answer did not require giving up useful conversation.
Her reasoning research tested whether successful answers reflected a procedure that a model could apply beyond familiar examples. Nouha was a co-first author of Faith and Fate, a 2023 study of multiplication, logic puzzles, and dynamic programming. The team represented problems as graphs of intermediate computations, allowing them to examine where solutions broke down. Models could perform well on familiar problem sizes yet fail when the required computations became deeper or wider. In those experiments, training on explicit reasoning steps did not reliably produce systematic generalization.
The co-authored RL Grokking Recipe later investigated whether reinforcement learning could develop algorithmic capabilities that pretrained models failed to execute. Using synthetic coding-problem families, the researchers observed some trained models move abruptly from near-zero reward to near-perfect accuracy after extended training. They also tested the reach of those gains: skills transferred within problem families and to some recombinations, while transformations requiring different strategies remained difficult. Learning a procedure and transferring it remained separate achievements.
At Ai2, Nouha worked as a postdoc and research scientist, co-leading safety and post-training efforts for the OLMo family of open language models. She led the creation of an OLMo safety suite, including an automated red-teaming framework and a safeguarding system presented at NeurIPS 2024. Her collaborative work examines both model behavior and how people respond to it.
At Cohere, Nouha studies how the organization of a coding-agent team affects what it can accomplish and what it costs. In her account of the team’s experiments, a lead that waited for its workers scored nearly ten points above a solo agent on ProgramBench under minimal instructions, but the team cost four to seven times as much. The researchers then asked which work justified that additional computation.
ProgramBench requires agents to reproduce a program’s behavior from documentation and an executable they can run but cannot read. Discovering interfaces and edge cases is therefore a substantial part of the task. Planning workers could probe different behaviors in separate contexts and return compact findings to the lead. A strong lead with 16 lower-cost planners approached the performance of a frontier-model team that delegated throughout planning, implementation, and verification, at roughly 70% lower cost than that all-stage team.
The useful division of labor changed with the task. On Commit0, repository structure and tests already supplied much of the plan, and implementation workers provided the greater benefit. Nouha’s position is to allocate agents to the stage that constrains progress, rather than assume that more delegation at every stage will improve results.
Coordination could also waste work the team had already paid for. A lead that continued coding while workers explored sometimes committed to a guess or submitted a solution before their findings arrived. Waiting prevented premature submission; explicit instructions about when to wait helped asynchronous leads catch up. Even with direct messaging available, workers sent 84% of messages to the lead, making its ability to use their findings central to performance.
Nouha does not treat that messaging result as evidence that peer communication is unnecessary. The harness divided work through a lead, and workers had limited visibility into who else was present. She identifies better communication and team organization as open research questions, especially as teams grow beyond what one lead can coordinate.
▶ Watch ↗31:22
Loading more talks and highlights…