Byung-Gon (Gon) Chun

Byung-Gon (Gon) Chun is the founder and CEO of FriendliAI and a professor of computer science and engineering at Seoul National University, currently on leave. He led the SNU research team behind continuous batching, a method for serving generative models that lets the system change its batch of requests between token-generation steps. Completed requests can leave and new requests can enter without waiting for the longest response in the original batch.

From mobile and cloud systems to AI infrastructure

Chun’s career spans networking, mobile computing, distributed data processing, and systems for machine learning. His education and early appointments took him from bachelor’s and master’s degrees in electronic engineering at Seoul National University to a computer-science master’s at Stanford and a computer-science doctorate at the University of California, Berkeley in 2007. After a postdoctoral appointment at the International Computer Science Institute, he joined Intel Research Berkeley in 2008.

At Intel, he worked on two collaborative projects addressing different problems raised by smartphones. CloneCloud explored splitting application execution between a mobile device and the cloud, allowing applications to use computing resources beyond the phone itself. TaintDroid tracked information flows to support real-time privacy monitoring on smartphones. His research record and honors include these projects, the EuroSys 2021 Test of Time Award, and the ACM SIGOPS 2020 Hall of Fame Award.

He subsequently joined Yahoo! Research Silicon Valley, where his work included Mobius, a unified messaging and data service for cloud-connected mobile applications, and workload-driven designs for big-data systems. He also led Yahoo’s systems research group before becoming a principal scientist at Microsoft. In 2013, he joined Seoul National University; later visiting appointments took him to Facebook’s Menlo Park research organization in 2016–2017 and Naver in 2020.

At SNU, Chun’s research expanded into the infrastructure used to train and serve machine-learning models. His collaborative publications include Apache REEF, a retainable evaluator execution framework, and Apache Nemo, a framework for optimizing distributed data processing. JANUS addressed how to execute imperative deep-learning programs through symbolic graphs, while Nimble focused on lightweight, parallel GPU task scheduling. These projects examined different parts of the same practical problem: how software can organize computation more effectively as workloads and available resources change.

Why generation needed a different scheduler

Chun founded FriendliAI with members of his SNU research team. He coauthored Orca, published at OSDI in 2022, with Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, and Soojeong Kim. The paper addressed a mismatch between conventional inference serving and autoregressive generation.

An autoregressive model produces an answer through repeated execution, generating one output token per iteration. Requests can therefore require different numbers of iterations. In the fixed-batch systems described in the paper, a request that finished early could not return to the client until the rest of its batch finished, and newly arriving requests had to wait for that batch to complete.

Orca combined two techniques to make serving respond to this pattern:

• Iteration-level scheduling, also called continuous batching. The scheduler asks the execution engine to run one model iteration on a batch at a time. Between iterations, the system can change which requests participate. A short answer no longer has to wait for the longest answer in its original batch, and a new request can join work already in progress.

• Selective batching. Orca applies batching to a selected set of Transformer operations so that batching and iteration-level scheduling can work together. The contribution includes both the decision about which requests run next and the execution strategy that makes those requests run efficiently.

The team implemented these techniques in a distributed serving system, with additional designs for scaling to models containing hundreds of billions of parameters. Chun’s role was part of a joint research effort spanning SNU and FriendliAI.

Bringing systems research into production inference

FriendliAI’s inference platform extends beyond continuous batching to custom kernels, speculative decoding, context-cache reuse, and reliable serving. These are company capabilities, rather than individual inventions attributed to Chun. Kernels optimize model operations; speculative decoding drafts and verifies tokens; cache reuse avoids recomputing previously processed context. Cache-aware routing, automatic failover, and autoscaling address how the service handles changing traffic and GPU failures.

That work matters for agents because a task can involve repeated model responses and tool calls. Delays accumulate across those steps, and a serving failure can interrupt the workflow. FriendliAI organizes its platform around keeping these repeated inference calls fast, efficient, and reliable in production.

The company’s May 2026 expansion into San Francisco brought it closer to U.S. customers, partners, and developers deploying open-weight and custom models. The announcement also described plans to grow its U.S. engineering and commercial teams and host developer events. For Chun, the company carries a long-running systems question into commercial AI infrastructure: how to turn better use of computing resources into faster responses and lower operating costs.