← All speakers

Dylan Patel

Conference affiliation: SemiAnalysis · 2026

Dylan Patel is the founder and chief executive of SemiAnalysis, the semiconductor and AI infrastructure research firm he began in 2020. He connects chip engineering with the economics of building and operating computing systems: why a faster accelerator may disappoint its buyer, how memory can constrain production, and why financing a cluster does not guarantee that electricity will be available to run it.

On this page

From hardware forums to SemiAnalysis

Dylan grew up around his family’s motel and gas-station businesses. Repairing a broken Xbox 360 sparked an interest in hardware that developed through years of reading, posting, and moderating online technology communities. He followed smartphones and graphics processors with an eye for both performance and commercial advantage. Although he bought an AMD graphics card for its price-performance ratio, he argued that Nvidia’s smaller chips, power efficiency, and stronger margins explained an advantage for the manufacturer that a consumer’s purchasing decision could miss.

Before SemiAnalysis, Dylan spent two years as a quantitative analyst at a small risk firm. In his account of the transition, he describes losing trust in his employer after a dispute over credit and compensation. During early 2020, he spent more time writing about semiconductors. After his previously anonymous identity was exposed online, he briefly stopped posting, then began publishing under his own name. SemiAnalysis launched with two substantial posts on his twenty-fourth birthday; consulting work followed.

For a period, he combined that work with a road trip through American national parks, working from inexpensive motels and reading textbooks about chips and AI. Technical conferences became another part of his education. At gatherings about lithography and photomasks, he initially understood little of what he heard. Reading, returning, and building relationships helped him connect published research with manufacturing practice: who supplied a chemical, how much a tool consumed, and where a shortage could interrupt production.

SemiAnalysis grew from a solo venture into a team covering manufacturing, accelerator architecture, cloud infrastructure, and model economics. Dylan edits much of its research and contributes to many reports. He describes its working culture as an argument between engineers excited by a technology and colleagues with financial backgrounds asking what it costs.

Benchmarks that reflect how people use computing

Dylan’s approach to benchmarking starts with a practical question: how much useful work does a system deliver for its cost? A machine’s advertised throughput leaves out software maturity, networking, electricity, and time lost to failures.

In their August 2025 H100 and GB200 training comparison, Dylan and Daniel Nishball examined performance alongside capital costs, energy consumption, downtime, and lost engineering time. Their ownership-cost estimates meant that GB200 needed to be roughly 1.6 times faster than H100 to offer better performance for the money. They also found substantial gains on existing H100 hardware as Nvidia’s software improved. For one GPT-3 training configuration, a year of software changes increased BF16 throughput by 57%.

The report paired criticism with a time-specific qualification: GB200’s early reliability and software problems limited its usefulness in August 2025, while the authors expected improvements as the architecture matured. They called for public benchmarks across cloud providers, broader testing with native PyTorch, and better diagnostics and acceptance tests for GB200 backplanes. These recommendations connected measurement to decisions buyers and operators could make.

SemiAnalysis’s InferenceX extends that approach to inference. In his explanation of the collaborative project, Dylan describes working with compute providers, chip companies, and serving-software developers to run repeated benchmarks and publish results, configurations, and deployable containers. Continuous testing addresses a weakness of one-time comparisons: models, drivers, and inference libraries change, taking performance with them.

Its central tradeoff is throughput versus interactivity. An interactive coding assistant may justify spending more to deliver tokens quickly to one user. Overnight document processing may benefit more from batching requests, increasing total throughput and lowering cost per token. Publishing optimized configurations across that range makes it harder to present one vendor’s best setup against another’s poorly tuned result. Cost and power measurements help readers choose a configuration for their own workload.

Matching the workload to the machine

  • Hardware-software co-design. Dylan argues that model architecture, kernels, communication patterns, and silicon need to be optimized together. In his case for co-design, he uses DeepSeek’s expert shapes and communication requirements to explain why an otherwise capable accelerator can perform poorly on a particular model. He also locates part of Nvidia’s ecosystem advantage in open models designed around GPUs: easier kernel development for competing chips does not erase those architectural choices. Specialization offers gains, but it can leave a multiyear hardware investment vulnerable to changes in model design.
  • Prefill and decode. Processing a prompt often demands substantial computation; generating successive tokens can depend heavily on memory bandwidth. Dylan’s coauthored Rubin CPX analysis follows that distinction into system costs. A machine devoted to prefill can underuse expensive high-bandwidth memory, making cheaper GDDR7 and simpler networking attractive. Separating prefill from decode also reduces interference between the phases. The tradeoff is flexibility: a fixed hardware ratio may become poorly matched as models and request lengths change.
  • Batching and context caching. In his 2024 inference analysis, Dylan explains how continuous batching admits new requests while others continue generating, sharing hardware across users. Context caching addresses another expense: a legal or contract-review application may repeatedly process the same long document even though each answer is short. Reusing its computed key-value cache avoids that work, but retained state consumes memory and may need to be stored in host memory or storage and moved back into use.

Following industrial bottlenecks—and qualifying the argument

Dylan applies these distinctions to competition between countries as well as companies. More logic-chip production does not automatically mean more working accelerators if memory, packaging, networking, or power remains unavailable.

His treatment of export controls shows a broad criticism becoming more specific. In his 2025 infrastructure talk, he sharply criticized enforcement and described routes through which Huawei obtained foreign chips and memory. The later September 2025 Ascend report, coauthored with AJ and Myron Xie, argued that controls were nevertheless constraining production through high-bandwidth memory. It also reported that one previously identified memory-supply route appeared to have stopped. Earlier stockpiling and enforcement failures could coexist with a later binding constraint. The authors advocated coordinated controls on memory and manufacturing equipment, with attention to the suppliers and packaging companies through which restricted components could pass.

Dylan’s infrastructure analysis also weighs Middle Eastern financing against American electricity constraints. He argues that overseas investment can fund capacity American AI companies would otherwise struggle to obtain, while acknowledging risks of diversion to restricted users. Providers bear a separate financial risk: they buy hardware and build facilities before rental income repays the investment. His support for expansion therefore includes a consequential condition—if the valuable work models can perform stops growing faster than computing capacity, the economics can turn against those building it.

Read the topics behind these talks

2 conference talks

Loading more talks and highlights…

References