← All speakers

Bio, Work & Ideas

Mahmoud Mabrouk

Conference affiliation: Agenta · 2026

Mahmoud Mabrouk is the co-founder and chief executive of Agenta, an open-source platform for building, evaluating, and improving AI applications and agents. His central concern is making AI systems reflect the judgment of the domain experts who understand whether they actually work.

Mabrouk began in academic machine learning, working on computational biology and protein-structure prediction, and earned a doctorate in computer science at Technische Universität Berlin. He subsequently joined a startup working with foundation models, where conventional machine-learning operations failed to accommodate changing prompts, expert evaluation, and production feedback. As he recalled about Agenta’s early years, he pursued an integrated, open-source platform despite investor skepticism about both its breadth and commercial model.

That platform grew to combine prompt management, evaluation, and open-source observability. In July 2026, Agenta 2.0 expanded into a collaborative agent workspace incorporating persistent context, feedback, permissions, human approval, and portable components such as tools, skills, and MCP servers.

  • Human-calibrated AI evaluation. Mabrouk maintains that automated judges accelerate development only when their decisions match informed human judgment. In his AI Engineer workshop, he separated an airline-support agent’s failures into policy compliance, response style, information delivery, and tool execution, favoring specialized binary judges with explanatory rationales over generic quality scores.
  • GEPA-based reflective prompt optimization. His public workshop materials explore improving evaluator prompts through mutation, merging, and Pareto-frontier selection. He emphasizes that results depend on annotation quality, seed prompts, reflection instructions, model choice, and inference costs; collectively covering every training example does not guarantee a single dependable evaluator.

Read the topics behind these talks

1 conference talk

Key ideas

Scroll to read ↓

A customer-support evaluator learns airline policy from annotated failures, revealing how seed prompts, reflection, candidate diversity and data quality shape reliable evaluation.

  • The monitor says everything is fine
    0:00 ↗
  • Evaluation determines how fast the application can improve
    1:18 ↗
  • A cancellation can succeed and still violate policy
    5:47 ↗
  • Derive the metrics from observed errors
    8:10 ↗
  • The explanation supplies the missing policy knowledge
    11:46 ↗
  • Generate candidates, test them, preserve complementary strengths
    14:32 ↗
  • The evaluator is the optimizer’s feedback channel
    19:44 ↗
  • Keep related tasks out of the validation split
    22:26 ↗
  • Start with a judge that does not invent violations
    26:06 ↗
  • Teach reflection to extract rules from failures
    29:09 ↗
  • Better accuracy does not mean the merge problem is solved
    32:46 ↗
  • Inspect one iteration before paying for a large search
    34:51 ↗
  • Supplying the policy and learning the policy are different starting points
    37:04 ↗
  • Spend the optimization budget where it can reduce recurring cost
    38:16 ↗

References