AI Engineer World's Fair 2024

Emergence Launch: AI Agents and the future enterprise

Satya Nitta· CEO and Co-founder, Emergence5:41

Loading slides…

Read the talk

Emergence Launch: AI Agents and the future enterprise

Emergence’s launch connects a planning-and-verification Orchestrator with Agent-E, a web agent, to turn delegated requests into coordinated software actions.

From a talk by Satya Nitta

Distributed systems meets AI research

What does it take to build AI agents that can carry out work across software systems? Emergence approaches that problem as both AI research and distributed systems engineering. Satya Nitta introduces an R&D-led company founded by a team from IBM Research, with colleagues drawn from Google Brain, Alexa, the Allen Institute of AI, Meta, and Microsoft.

Nitta cites the team’s experience with IBM Watson, Alexa, and recommendation engines behind Amazon Prime, Amazon Video, Twitter, and Eventbrite. That deployment background explains the company’s emphasis: develop the science of agents while building infrastructure that lets other developers connect them into working systems.

0:170:35
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:17 · section reference included

From answers to actions

The ambition is older than today’s language models. Nitta traces it to science fiction of the 1940s and 1950s: AI would perform actions for people. Here, that means operating browsers, enterprise systems, and other software to automate complex human workflows. The desired output is work performed through software, with productivity benefits for both consumers and enterprises.

Slide stating that AI agents will operate most of the world’s software, automate complex human workflows, and unlock the full promise of AI.Open full source frame
AI agents operating software to automate complex human workflows.

Enterprise is Emergence’s focus because its workflows are demanding. Coordinating work across business software is expected to stress the limits of autonomous AI, making enterprise applications both a commercial destination and a proving ground for the research.

1:091:20
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:09 · section reference included

An agent that coordinates other agents

Emergence introduces two infrastructure platforms, starting with the Orchestrator. At the time of the recording, Nitta says early access is live and forecasts general availability in August. Those are the launch’s contemporaneous availability claims.

The Orchestrator is itself an agent. Its operating loop has three parts:

  1. Plan the work.
  2. Act to carry it out.
  3. Verify the result.

Nitta also describes memory and improvement over time. The purpose of this loop is to coordinate multiple agents and stitch their contributions into a larger workflow. Verification is therefore part of the agent’s job, rather than treating an issued action as the end of the task.

The scope ranges from choosing among language models to coordinating business processes:

ScopeOrchestrated componentsIntended work
SimpleGeneralist and open-source LLMsCoordinate model use
ComplexMultiple agentsEnterprise workflows such as claims processing

The difference is the work being composed: model responses at the simpler end, contributions from several agents toward a business outcome at the more complex end.

1:572:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:57 · section reference included

Giving the Orchestrator a web agent

An orchestrator’s capabilities depend on the agents it can call. Emergence is integrating the Orchestrator with Agent-E, an open-source project intended to understand and operate web interfaces as a human would. Nitta points to an earlier session by his colleague Tamer for more on that component. The division of responsibility is straightforward: the Orchestrator coordinates the workflow; Agent-E supplies the ability to act through the web.

Agent-E slide describing understanding and operating web interfaces like a human, with a GitHub logo and QR code.Open full source frame
Agent-E: an open-source web agent for enterprise workflows.

Nitta reports that Agent-E tops the WebVoyager benchmark. The talk supplies neither a score nor evaluation conditions, so its description of Agent-E as the best web agent should be read as a benchmark-specific company claim. The intended product role is broader: support multiple enterprise workflows while working with the Orchestrator.

2:513:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:51 · section reference included

Developer controls and a delegated reservation

The product video adds a developer-facing layer around this coordination. Generalist and task-specific LLMs and agents are described as operating with appropriate guardrails, though the video does not explain their implementation. Its dashboard capabilities fall into three groups:

  • Prompt analysis: inspect prompts used by the system.
  • Cost and latency optimization: tune the economics and responsiveness of model use.
  • Model development: create models with your data and enhance existing models.

The video also promises on-demand migration to newer LLMs and describes the Orchestrator as improving with use. Agent-E, still described as being in R&D development, appears as one connected component in the LLM and agent garden.

Diagram labeled LLM & Agent Garden lists Emergence-1, Agent-E, Custom-Agent, ChatGPT, Meta, and Gemini, with Agent-E highlighted.Open full source frame
The Orchestrator connects to Agent-E in an LLM and agent garden.

The concrete example is a delegated restaurant booking: find a reservation for 15 people near the office around 6 PM, considering Asian, Mediterranean, or Mexican food, then send the details to Levi. As a structured task record, the request contains both booking constraints and a communication requirement:

json

{
  "reservation": {
    "partySize": 15,
    "location": "near the office",
    "time": "around 6 PM",
    "cuisines": ["Asian", "Mediterranean", "Mexican"]
  },
  "followUp": {
    "recipient": "Levi",
    "content": "reservation details"
  }
}

This is a requested outcome, not a shown completed transaction. Its significance is the composition of work: finding an option that satisfies several constraints, securing a reservation, and communicating the result are all part of the delegation. The video closes by identifying planning and self-improvement as core research problems behind that ambition.

3:474:05
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:47 · section reference included

Self-improvement and agent-oriented programming

Nitta closes by making self-improving agents the labs’ central research focus. Planning and reasoning are supporting directions, alongside agent-oriented programming: ways to stitch agents together so their combined behavior can accomplish more complex work. He names the direction without presenting a programming language or composition API.

That research remains anchored in enterprise applications, including robotic process automation (RPA) and document processing. These are the contexts in which better planning, reasoning, and agent composition are meant to become useful systems that perform work.

4:525:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:52 · section reference included

Resources

From the talk

  • Open-source browser automation agent with installation instructions, natural-language commands, and configuration examples.

  • WebVoyagerPaper3:18

    Introduces a web agent, tasks across 15 real websites, and an automated evaluation protocol.

Updates since the talk

Transcript download
Full transcript

Loading transcript…