AI Engineer Summit 2025

Where AI is superhuman: The right jobs to automate with LLMs

Andy Triedman· Partner, Theory Ventures11:53

Loading slides…

Read the talk

Where AI is superhuman: The right jobs to automate with LLMs

The strongest automation opportunities often arise where people already rely on rules to handle overwhelming volume, from security alerts to customer messaging.

From a talk by Andy Triedman

Which jobs actually suit automation?

You can ask an LLM to act as a personal trainer, a primary care doctor, or an editor, and it will attempt all three. Andy Triedman, a partner at Theory Ventures, describes using assistants in each of those roles. Businesses face the same abundance of possible applications across sales, customer service, security, finance, and engineering. But willingness to attempt a task does not tell a founder where to build or an executive where to invest. The nature of the work determines the opportunity.

Slide asking, “You can try applying AI to any job... but where does it actually make sense?” with the presenter inset at upper right.Open full source frame
Where does applying AI to a job actually make sense?

Theory approaches that selection problem as an early-stage investor in data and machine learning infrastructure and applications. Based in San Francisco, the firm conducts thematic research, including conversations with hundreds of buyers and builders across job categories. The useful unit of that research is a workflow: what information comes in, what decisions must happen, and what prevents the work from getting done today.

0:000:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

Transformation, synthesis, and reasoning

Triedman starts with an account of LLMs as models of the distribution of their training data. For workflow automation, three capabilities emerge from that foundation. The first is transformation: converting a PDF into a spreadsheet, a spreadsheet into an email, or an email into JSON. Historically, integrating and transforming information has blocked many automation projects. Triedman sees LLMs as effectively solving that bottleneck by making information usable across formats.

An email-to-JSON task makes that boundary concrete. For an illustrative email reading “Please send the replacement to 14 Oak Street; order A123,” the useful output is a record that another system can consume:

json

{
  "order_id": "A123",
  "request": "replacement",
  "shipping_address": "14 Oak Street"
}

The transformation extracts the request; it does not decide whether the replacement should be approved. That distinction separates moving information into a usable form from making a business decision.

The second capability is synthesis: reducing a large body of information into a summary or an answer. Theory uses deep research platforms from Gemini and OpenAI to synthesize hundreds of websites when investigating a technology or market.

The third is reasoning, where the availability of training data becomes especially consequential. Written reasoning leaves out much of what people know: assumptions, organizational context, and intermediate judgments that nobody records. Models have substantial material to learn from in common sense, Stack Overflow, support forums, and increasingly code, mathematics, and logic that can be generated programmatically. The reasoning of a security analyst, lawyer, or accountant handling a complex case is less readily available. Capturing that domain knowledge can therefore become a differentiator for a company building automation, rather than something it can assume the base model already possesses.

1:071:19
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:07 · section reference included

Break the job into tasks

A job title is too broad to specify an automation. Security analysis becomes more tractable when it is decomposed into an iterative procedure:

  1. Receive an alert.
  2. Run a query or research the event to collect more information.
  3. Transform or synthesize the result.
  4. Analyze the available evidence.
  5. Return to research if necessary, continuing until there is enough information for a determination.

The loop matters: an investigation is not necessarily one retrieval followed by one answer. Its next information request depends on what the analyst has already learned.

Security analysis flowchart linking an alert to research, synthesis or transformation, analysis of existing information, and a final determination, with an arrow returning from analysis to research.Open full source frame
Breaking security analysis into tasks and an iterative research loop.

Even the query step contains smaller tasks: choose a tool, write a query, and debug errors. Those subtasks may belong to different people or to automated systems configured by people. Conversely, a person's role includes more than the investigation itself. Meetings, coordination, and interpersonal responsibilities remain part of the job even when its core task can be automated. Task coverage and role replacement are different questions.

2:402:51
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:40 · section reference included

Match the system to volume and complexity

Work differs along two useful dimensions: how many tasks arrive and how complex each task is.

WorkTypical constraint
Strategic planningThinking and coordination around a few complex decisions
Customer supportRepeated cases requiring customer reasoning and internal queries
CRM prospect queues or SIEM alertsLarge volumes of relatively simple tasks
Supply-chain messagingHundreds of repetitive emails or messages

A SIEM is a security information and event management system; a CRM manages customer relationships. In both, the task queue makes the volume problem visible.

Different positions on that spectrum call for different operating models:

  • Copilots for complex work. Humans remain in control and delegate selected tasks. Better models do not automatically acquire the unwritten context, priorities, and reasoning behind strategic decisions.
  • Core workflow automation. The system drives substantial or complete workflows, while experts configure its behavior and people review individual outputs or handle escalations.

Across the areas Theory has researched, Triedman estimates that many jobs could have 40–70% of their day-to-day work automated. This is an estimate about portions of work, not evidence that the corresponding roles disappear. The more radical opportunity lies farther along the spectrum, where volume itself dominates the job.

3:303:46
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:30 · section reference included

When the competitor is rules-based software

High-volume, relatively simple work is often difficult because there is too much of it. Once a team is overwhelmed, people stop handling every case individually. They write rules and build workflows to process the excess. The practical competitor for AI is then the existing rules-based system, rather than a human giving every case full attention.

Triedman contrasts a perception of LLMs as 80–90% as reliable as humans with a potential 10× advantage in work constrained by volume. This is a conditional scale argument, not a measured benchmark: the system must first be capable of the task, after which repeating it 10, 100, or 1,000 times a day becomes the source of advantage. His two examples—security investigations and customer engagement—come from Theory's portfolio.

5:135:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

5:13 · section reference included

Dropzone AI: investigating the alerts people cannot reach

As a company buys more security products, those products generate more alerts. Someone must investigate whether each alert represents a real threat requiring remediation or a false positive. Dropzone AI addresses the resulting investigation backlog. Triedman says analysts might inspect only 1% of alerts and use rules to filter the remainder. Rules handle simple cases, but the number of workflows to maintain grows as the team tries to cover more edge cases.

Repetition also creates a staffing problem. Triedman reports that analysts typically leave the role after 12–18 months. He also cites a shortage of about four million; the matching ISC2 workforce estimate concerns the global cybersecurity profession's gap between staffing and perceived need, rather than four million unfilled analyst positions.

Dropzone's agentic systems perform end-to-end investigations. Triedman characterizes their strengths as expertise across query languages, typo avoidance, and near-perfect memory. Those are claims about the system's advantages, not an error-free guarantee; his companion account acknowledges hallucinations and describes typo reduction probabilistically. The more consequential point is coverage: humans already lack time to investigate the full queue, so matching an ideal analyst on every dimension is not the threshold for improving the existing workflow.

Triedman describes Dropzone as providing investigation coverage 24 hours a day, 365 days a year. He also describes shared learning across customers: when a new threat appears, the network can make that knowledge available so other customers can identify and block it. The claimed advantage combines continuous investigation with reuse of discoveries beyond the customer where they first occurred.

6:026:18
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:02 · section reference included

Aampe: learning what to send each customer

Customer engagement has a similar scaling problem. Apps, subscription services, and retailers reach users through texts, notifications, and in-app personalization. Choosing a message for one person may be straightforward; choosing for a million people is not. Marketing teams therefore construct rules-based journeys, such as a sequence for new users or a reminder for someone who abandoned a cart. Aampe is Theory's example of automating this work.

Those journeys become difficult to maintain because users differ across several dimensions at once: interests, preferred messaging style, channel, and time of day. A team can manage a few cohorts, but writing branches for all combinations creates a combinatorial explosion. Measurement adds another limitation. A click is easy to associate with a message; a purchase a week later or subscription retention a month later is harder to connect to the strategy that influenced it.

Aampe changes the marketer's role from journey designer to experimentalist. Marketers create hundreds of variants, while agents explore what to send, to whom, when, and through which channel. The system evaluates effects across users' subsequent activities. The accompanying investment account describes reinforcement learning for this experimentation; the mechanism should not be understood as an LLM simply generating each personalization decision.

Aampe slide showing a branching journey interface on the left, a large right-pointing arrow, and an “AI Agentic Market Formation” diagram on the right.Open full source frame
Aampe contrasts a branching customer journey with an agentic system diagram.

Triedman reports substantial improvements in satisfaction, engagement, transactional outcomes, and retention, without providing numerical results or experiment details. His explanation is individual personalization: a person's experience need not be determined by membership in a large shared bucket.

Continuous experiments can also reveal customer groups the business did not know existed. Triedman describes a food-delivery customer discovering users who responded only to messages at 11 p.m. or midnight—later than the company would ordinarily contact them. These late-night snackers were not a cohort the marketing team had specified in advance. Their behavior emerged through experimentation, producing knowledge that data science and product teams could use beyond the messaging campaign itself.

7:217:37
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:21 · section reference included

From doing tasks to operating systems

Automating a workflow changes the surrounding team, not just the person who previously performed a task. Triedman identifies three directions for the remaining work:

  • Review outputs. Approve actions and handle cases escalated by the system.
  • Maintain the automation. Update workflows, model instructions, and data systems to improve performance and reliability.
  • Do higher-order work. Spend time on strategy and other responsibilities previously displaced by daily operations.

All three require experience in the domain. Knowing how to review an investigation or improve its instructions depends on understanding the work being automated.

That requirement conflicts with the staffing shape of many high-volume functions. Today, a large base of junior individual contributors executes daily tasks beneath a smaller group of experienced staff. Theory forecasts that substantial automation will make these organizations smaller and shift the remaining positions toward advanced or managerial work. The resulting structure could resemble a diamond or an inverted pyramid. These are forecasts about organizational change, not reported staffing outcomes.

Slide with a Roles column pointing from “Doing tasks” to reviewing LLM outputs, maintaining LLM systems, and strategic work; a Team structure column points from an upright triangle to a diamond and an inverted triangle.Open full source frame
Roles shift toward reviewing outputs, maintaining LLM systems, and strategic work as team structures change.

The difficult consequence is the training path. If fewer entry-level roles remain, how does an organization hire and develop the people who will eventually possess the expertise needed to review and maintain its systems? Automating the tasks that teach a profession can remove part of the route into that profession. Triedman leaves that as an open organizational challenge.

9:339:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:33 · section reference included

Technical suitability is only part of the business

Task decomposition reveals what a model must do; volume and complexity reveal where doing it repeatedly could change the economics of the work. The surrounding organization then determines how that capability can be adopted, supervised, and maintained. Those questions matter both to founders selling automation and to executives buying it.

But technology–problem fit is not business viability. A system can be well suited to a workflow without supporting a good business. Triedman's final qualification is to examine the severity of the pain, alignment of incentives, market size, and market structure as well. Being able to automate the work establishes a capability; it does not by itself establish who will pay, why they will adopt it, or whether a durable company can be built around it.

11:0011:15
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:00 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…