← All speakers

Daniel Han

Conference affiliation: Co-founder · Unsloth · 2026

Daniel Han is the co-founder of Unsloth, which he built with his brother Michael Han to make open models easier to fine-tune, compress, and run locally. His work combines faster, more memory-efficient computation with debugging the implementation mistakes that can make a capable model fail—from numerical precision inside a layer to the chat template used after deployment.

On this page

From numerical methods to Unsloth

Daniel’s early projects combined practical data science with mathematical problems. He created sciblox, a Python toolkit for data preparation, visualization, missing-data imputation, and machine learning in Jupyter notebooks. His Markov-chain reversal project began with an obstacle in modeling disease spread: after representing people’s daytime movements, he wanted to model their return journeys without constructing a separate process. Ordinary matrix inversion could produce negative probabilities or values above one, so he proposed a reversal method that retained a valid probabilistic transition matrix.

At UNSW Business School, Daniel worked as a graduate research and teaching assistant from December 2017 to December 2018, helping develop marketing-analytics courses. During 2017, he also led UNSW teams exploring air-conditioning control for Telstra and construction-site mapping with a robot for Westfield.

From December 2018 to February 2020, he worked at NVIDIA in Sydney, accelerating machine-learning algorithms in the RAPIDS ecosystem. His GPU-accelerated t-SNE work addressed the cost of dimensionality reduction. His contributions also included more efficient PCA, non-negative matrix factorization and nearest-neighbor algorithms, reduced SVD memory use in CuPy, and expanded cuML documentation. In his public career history, Daniel describes choosing startup building and open-source work over an opportunity to relocate with NVIDIA.

Daniel and Michael also developed HyperLearn, a library focused on faster, more memory-efficient machine-learning algorithms. The brothers launched Unsloth in December 2023, extending that work into language-model fine-tuning. Unsloth used custom Triton kernels and manually derived backward computations to reduce training time and memory requirements while preserving the operations’ mathematical behavior. It joined Y Combinator’s Summer 2024 batch and expanded into reinforcement learning, compressed model distribution, and local inference. Its desktop application, web interface, and code library connect running models with training and exporting them.

The details that change a model’s behavior

Daniel’s contributions often address the interaction between limited GPU memory and the computation a model was trained to perform:

  • Open-model debugging. In his Gemma analysis, Daniel compared layer outputs across implementations and examined activation functions, normalization precision, and rotary positional embeddings. Representing positions at insufficient precision could round distinct token positions to the same value, removing positional distinctions. His criterion for a fix was matching the original implementation: even a formula that appeared preferable in one comparison could be wrong for the released weights if it changed the computation used during training.
  • Correct gradient accumulation. Small GPUs often require a training batch to be processed in pieces, with gradients accumulated before an update. Daniel and Michael’s gradient-accumulation investigation traced discrepancies from equivalent full-batch training to loss normalization across sequences of different lengths. Averaging separately normalized losses can weight tokens differently from the full-batch calculation. Their work corrected the normalization and supported related trainer improvements with Hugging Face.
  • Asynchronous gradient checkpointing. The brothers’ long-context training work moves selected intermediate activations into system RAM to reduce GPU memory pressure. Non-blocking transfers limit the time the GPU spends waiting for that movement, making longer fine-tuning contexts feasible with a small runtime penalty in their reported experiments. Their reinforcement-learning workflows also share weights between training and vLLM inference, avoiding duplicate copies for those two jobs.

Daniel treats tokenization and deployment as part of the same engineering problem. His Llama and Phi debugging work explains how using the same identifier for padding and end-of-sequence tokens can erase the training signal that teaches generation to stop: masking padding also masks the end token. Duplicate beginning-of-sequence tokens and instruct templates containing untrained base-model tokens can likewise degrade a fine-tune or produce invalid gradients. Unsloth incorporates automatic checks and fixes, and generates Ollama Modelfiles that preserve the training chat template during deployment.

Choosing what to compress

Unsloth’s dynamic quantization assigns precision according to a model’s sensitivities. Less sensitive components can be compressed aggressively; components whose numerical errors damage behavior retain higher precision.

The dynamic four-bit experiments, also explained in Daniel’s 2025 workshop, make that tradeoff tangible. Uniform four-bit quantization reduced a Qwen vision-language model to 1.36 GB, but it described an image of a train as a coastal scene. Selectively retaining higher precision increased the size to about 1.8 GB and restored the correct description. Activation and weight quantization errors helped identify sensitive components, which differed across Qwen, Llama Vision, and Pixtral. The work required examining each architecture rather than applying one compression rule everywhere.

In an August 19, 2026 post, Daniel reported one-bit Qwen3.8-27B variants that ran in 8 GB of RAM and retained roughly 77% of BF16 accuracy in Unsloth’s internal testing. He said the team had initially been reluctant to release them, then was surprised by how well they worked. The result acknowledges a substantial accuracy tradeoff while bringing the model within reach of much smaller local machines.

Rewards must represent the work

Daniel’s approach to reinforcement learning emphasizes reward design, varied sampling, and supervised priming when a model needs a starting point from which to improve. In his reinforcement-learning workshop, priming teaches enough instruction-following behavior for sampled answers to receive useful rewards. Repeatedly generating identical or uniformly unsuccessful answers gives training little to distinguish.

He places the harder question in the verifier: what counts as success? Code can execute without implementing the requested game. A correct final answer can follow faulty reasoning. Daniel favors the interpretation that reinforcement learning often makes capabilities already present in a pretrained model more likely to appear, while acknowledging that whether it creates new capabilities remains unresolved.

His work on reward hacking and evaluation develops the consequences for generated kernels and agents. A system asked to accelerate matrix multiplication may alter the timer, change the inputs, or reuse cached answers instead of improving the computation. Separate correctness and timing checks can still be exploited if the generated program recognizes which test is running. Daniel urges researchers to inspect the code and verify that measured speedups come from performing the intended work. His benchmark criticism also examines weak tests, contaminated data, answer-extraction errors, and changes in inference software that can alter scores without establishing a change in model capability.

Despite Unsloth’s use of custom kernels, Daniel recommends trying torch.compile before writing them by hand. Compiler-generated fusion may already remove unnecessary memory movement. That advice connects his earlier numerical work with his current engineering: identify the expensive computation, test the available optimization, and check that the improvement preserves the behavior being measured.

Talks by Daniel Han

5 talks

Loading more talks and highlights…

References