Merve Noyan is a machine-learning engineer on Hugging Face’s ML advocacy engineering team, a coauthor of SmolVLM, and a coauthor of the O’Reilly book Vision Language Models. She develops open multimodal models and practical tooling that make image understanding, customization, and local deployment accessible with limited computing resources.
Earlier in her career, Noyan built natural-language-understanding systems for information retrieval and conversational agents. At Hugging Face, she expanded into computer vision, multimodal learning, and open-source developer infrastructure. Her work includes a 2024 guide to vision-language models and direct preference optimization for vision-language models in TRL, which trains multimodal systems using preferred and rejected responses without a separate reward model.
Noyan coauthored the SmolVLM release and research paper, examining how visual encoders, image tokenization, and training data affect the performance and memory demands of compact vision-language models. The family includes 500-million- and 256-million-parameter variants; the smallest requires less than one gigabyte of GPU memory for inference. Her smol-vision repository provides recipes for shrinking, optimizing, and customizing visual models, while her SigLIP projects explore image-text representations and their integration with Transformers.
Open weights do not guarantee open models. Noyan rejects the assumption that publishing weights on Hugging Face automatically makes a model fully open. Licensing, accessible implementation, and control over fine-tuning and deployment determine whether developers can adapt systems commercially, inspect their behavior, and protect sensitive data.
Small multimodal models demand deliberate design. SmolVLM treats architecture, visual tokenization, training data, memory, and throughput as connected engineering decisions. Quantization expands access to larger models, but quality and latency remain genuine tradeoffs; purpose-built smaller models can bring visual understanding to consumer devices.
Visual document retrieval preserves crucial context. Noyan advocates retrieving document pages as images with systems such as ColPali, then passing relevant pages to a vision-language model. This approach retains layouts, charts, tables, and images that brittle OCR-heavy pipelines can discard.
Agent-driven training should absorb operational complexity. Through agent skills and Model Context Protocol, coding assistants can discover models and datasets, estimate hardware requirements, launch fine-tuning jobs, and return trained models to Hugging Face Hub. Noyan has also developed computer-vision skills for object detection and segmentation, reducing setup problems such as incompatible bounding-box formats.
Making a model fit is only the beginning: selective precision, architecture-aware evaluation and runtime compression determine whether local intelligence remains useful.
Open models, agent traces, skills and managed compute turn a coding assistant into a tool for selecting models, launching training and processing research papers.
What would it take to put an AI engineer at your fingertips?
