AI Engineer Summit 2025

Lessons from Building LinkedIn's GenAI Platform

Xiaofeng Wang· Engineering Manager, Generative AI Foundations, LinkedIn17:53

Loading slides…

Read the talk

Lessons from Building LinkedIn's GenAI Platform

LinkedIn’s path from generated articles to recruiting agents shows how product needs shape orchestration, memory, observability, and the team responsible for them.

From a talk by Xiaofeng Wang

Before you start: Familiarity with LLM prompts, APIs, and basic Python will help; retrieval, memory, and agent orchestration are explained as they arise.

A prompt, an article, and two technology stacks

LinkedIn’s Collaborative Articles began with a simple interaction: generate an article, then invite members to contribute their expertise. Xiaofeng Wang describes this 2023 product as LinkedIn’s first formal GenAI feature and identifies GPT-4 as the model used to produce the long-form content. From the application’s perspective, the mechanism was straightforward: a prompt went in and a string came out.

Supporting that interaction required a gateway to centralize model access and Python notebooks for prompt engineering. But development and serving lived in different worlds: Python on the backend and Java in the online application. These were useful components, not yet what Wang would call a platform. The initial architecture solved access to generation; it did not yet provide a shared way to build richer AI experiences.

Slide pairs a Collaborative Articles screenshot with six boxes: LLM Playground, Shared Java Midtier, GAI Gateway, Prompt Python Notebooks, Spreadsheet for Eval, and Prompt Resolution Lib.Open full source frame
Collaborative Articles alongside the early GenAI application platform components.
1:111:26
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:11 · section reference included

Personalization makes shared infrastructure necessary

The limitation became apparent when products needed LinkedIn’s own data. By mid-2023, the next generation of experiences—internally called Copilot or Coach—needed to answer personalized questions. Consider job fit: the application looks at a member’s profile and a job description, then uses retrieval-augmented generation, or RAG, to provide recommendations about whether the member fits the role. A generic article prompt cannot supply that context by itself.

The team built a Python SDK on top of LangChain to orchestrate LLM calls and integrate LinkedIn’s infrastructure. Developers could assemble an application through this shared layer instead of connecting each infrastructure service themselves. This also motivated unifying the technology stack: transferring prompts developed in Python into Java was costly and introduced errors.

Two additional capabilities made the SDK more than a wrapper around model calls. A prompt source of truth versioned prompts and provided structure for meta prompts. Conversational memory tracked LLM interactions and retrieved content so that the application could inject that context into subsequent interactions. Prompt management controlled the instructions; memory supplied the evolving context needed for a conversational product.

2:282:47
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:28 · section reference included

Recruiting agents need execution and discoverable skills

The next product step was LinkedIn Hiring Assistant, which Wang describes as the company’s first real multi-agent system. It launched in charter in 2024, initially for selected customers and LinkedIn recruiters. Wang frames the recruiting workload broadly: posting jobs, evaluating hundreds of candidates, and reaching out to them. The system’s purpose was to automate tedious work across that process, rather than merely return advice in a conversation.

That change expanded the Python SDK into a distributed agent orchestration layer. It had to handle distributed execution, retry logic, and traffic shifting. An agent that chooses and performs actions needs more operational support than an application that makes a single model request.

Actions also require a usable interface to enterprise APIs. The team built tools for developers to publish their APIs into a centralized skill registry. The registry addressed both discovery—finding an available skill—and invocation—calling it from an application. Publishing an API as a skill made the existing service accessible through the agent platform’s common machinery.

4:314:55
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:31 · section reference included

Remember interactions and inspect autonomous behavior

Memory evolved alongside execution. Beyond retaining conversational content, experiential memory extracted, analyzed, and inferred contextual knowledge from interactions between users and agents. The platform organized memory into working, long-term, and collective layers. The distinction is useful: preserving an interaction gives the system a record; extracting contextual knowledge makes that interaction useful as experience.

Autonomy made observability essential. Because an agent can decide which API or LLM to call, its behavior is difficult to predict in advance. LinkedIn built an in-house solution on OpenTelemetry to capture fine-grained telemetry that could be used to replay agent calls. An analytics layer on top of those records then helped guide future optimization. The platform needed visibility into the agent’s execution, not just its final response.

6:116:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:11 · section reference included

One interface to a larger AI ecosystem

These investments formed four main areas of the GenAI application platform:

AreaResponsibility
OrchestrationCoordinate model and agent execution
Prompt engineeringManage instructions and their versions
Tools and skillsSupport discovery and invocation
Content and memorySupply context across interactions

The application platform brought these concerns together, but it did not own the entire AI ecosystem.

Sister teams handled model work such as fine-tuning open-source models, responsible AI policies and standards, and the machine-learning infrastructure that hosted models. The GenAI platform’s role was to give application developers a unified interface to that ecosystem. Developers could use those capabilities without first learning every underlying service and integration.

Wang gives model selection as a concrete example: in LinkedIn’s SDK, changing one parameter in one line of code could switch an application from an OpenAI model to an on-premises model. Developers still had to do prompt engineering for the selected model; the abstraction removed infrastructure integration work, not differences in model behavior. Centralization also created a place to enforce best practices and governance, helping developers build applications efficiently and responsibly.

Layered diagram covers orchestration, prompt engineering, tools, models, responsible AI, AI platform, and context and memory. Five bullets describe integration, developer experience, reliable development, and responsible AI practices.Open full source frame
LinkedIn’s GenAI ecosystem and the platform’s integration and governance responsibilities.
7:377:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:37 · section reference included

The boundary between optimization and serving moves

The platform emerged piece by piece, which raises a reasonable question: with so many vendors offering GenAI tooling, does a company need this platform capability, and should it build or buy it? Wang’s answer addresses the need for the capability rather than selecting a vendor or prescribing that every component be built internally. The reason is a change in how AI applications are developed.

In the traditional arrangement Wang describes, model optimization and serving have a relatively clear boundary. AI engineers and product engineers can use different stacks and need not work in the same codebase. In a GenAI application, that separation fades: engineers on either side can change the overall system’s performance.

Development concernTraditional model-centered systemGenAI application
Optimization and servingSeparate phasesIntertwined work
Engineering stacksCan remain separateNeed shared tooling
System improvementDivided by roleSpans both groups

Once optimization reaches into application behavior, tooling and engineering practices have to support both groups working on the system.

Wang connects this to Berkeley AI Research’s concept of a compound AI system: a system that tackles AI tasks through interacting components, including multiple model calls, retrievers, or external tools. Improving such a system requires skills that cross the traditional AI-engineering and product-engineering boundary. A shared platform bridges that gap by making the components usable through common interfaces and practices.

9:289:41
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:28 · section reference included

Build the team’s range through collaboration

That platform remit creates a demanding hiring profile. Looking at the strongest engineers on his team, Wang identifies infrastructure integration skills, a developer product-management sensibility for interface design, AI and data-science knowledge, and the ability to learn new techniques while remaining hands-on. Finding all of these in one person is rare—rare enough that he jokes the candidate would be worth more than a unicorn. Hiring therefore requires deliberate trade-offs.

  • Core engineering skills: Prioritize strong software engineering over existing AI expertise.
  • Potential: Hire for the ability to learn rather than relying mainly on experience or degrees, because the field changes quickly.
  • Complementary backgrounds: Build the missing range across the team. LinkedIn’s mix included full-stack engineers, data scientists, AI engineers, data engineers, recent research-university graduates, and people with startup experience.

The development mechanism was joint project work. Wang reports that strong engineers acquired complementary skills through collaboration and grew toward the broader ideal profile.

Dream Team slide shows an ideal engineer profile beside four bullets: Focus on Core Skills, Hire for Potential, Promote Diversity, and Foster Collaboration.Open full source frame
Building a team through core engineering skills, potential, diversity, and collaboration.

Critical thinking matters after the team is hired, too. Wang describes a recurring team premise: what they are building may become outdated within a year, or even in less than six months. That premise drives an ongoing practice of evaluating open-source packages, talking with vendors, and proactively deprecating internal solutions. Building the platform does not require defending every component indefinitely.

11:3411:54
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:34 · section reference included

Choose Python and treat prompts as production parameters

Returning to the technology stack, Wang recommends Python where possible. LinkedIn began with Java and Python, debated the choice internally, and ultimately chose Python because of its alignment with research and the open-source ecosystem. Wang also reports that Python proved scalable in LinkedIn’s experience; he supplies no workload measurements for a language comparison.

Prompts deserve the operational discipline applied to model parameters. A prompt source of truth needs robust version control because changing instructions changes application behavior. An accidental production edit can create unwanted side effects even when the surrounding application code remains unchanged.

A small Python example makes the distinction between authoring a prompt and selecting a production version concrete. Here, the application explicitly selects job-fit-v1; adding job-fit-v2 to the catalog does not change that selection:

python

from types import MappingProxyType

PROMPTS = MappingProxyType({
    "job-fit-v1": (
        "Assess this member's fit for the job using the supplied facts.\n"
        "Profile: {profile}\nJob description: {job_description}"
    ),
    "job-fit-v2": (
        "Compare the supplied profile with the job requirements.\n"
        "Separate supported matches from missing information.\n"
        "Profile: {profile}\nJob description: {job_description}"
    ),
})

PRODUCTION_PROMPT_VERSION = "job-fit-v1"


def render_job_fit_prompt(profile: str, job_description: str) -> str:
    template = PROMPTS[PRODUCTION_PROMPT_VERSION]
    return template.format(
        profile=profile,
        job_description=job_description,
    )

This illustrates explicit version selection, not LinkedIn’s internal SDK. The production safeguard depends on preserving released versions and making changes to the selected version deliberate.

The other essential investments follow the product’s needs. Memory injects rich enterprise data into agent experiences. Tooling around existing APIs turns them into skills that agents can call easily. Together, these components give applications controlled instructions, relevant context, and access to actions.

14:3014:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:30 · section reference included

Grow from immediate needs into developers’ workflows

The route to adoption starts smaller than the finished architecture suggests. Solve an immediate need instead of building a full platform at the outset. LinkedIn began with a simple Python orchestration library and expanded into the broader set of components as product requirements grew. The useful unit of progress was a capability developers could use, not completion of the entire platform diagram.

Scaling also meant reusing infrastructure. LinkedIn used its existing messaging infrastructure as a memory layer, an approach Wang describes as cost efficient and scalable. The lesson is to look for infrastructure that already provides the foundations a new capability needs, rather than assuming that an AI-specific requirement demands an entirely new storage system.

Developer experience determines whether those capabilities are adopted. The platform exists to make developers productive, so its technology needs to fit their existing workflows. A technically capable system that makes everyday development harder misses the reason for building it.

Full-screen slide with three blue columns labeled Start Simple, Evolve Gradually; Focus on Scalable Solutions; and Prioritize Developer Experience.Open full source frame
Platform adoption lessons: start simple, build for scale, and prioritize developer experience.

For the lower-level implementation details, Wang points to the team’s engineering article on LinkedIn’s GenAI application stack, by Karthik R., Xiaofeng Wang, and Sandeep Jha.

16:0216:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:02 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…