← All speakers

Bio, Work & Ideas

Suman Debnath

Conference affiliation: Amazon Web Services (AWS) · 2025

Suman Debnath is director of developer relations and product at Crusoe, specializing in AI infrastructure, managed inference, and the engineering required to run advanced models reliably. His work spans distributed storage, machine learning, visual document retrieval, open-source agents, and production AI architecture.

Debnath began his career in systems, storage, and performance engineering, with roles at Toshiba, Broadcom, and NetApp. At Amazon Web Services, he worked on Amazon Elastic File System and became a principal developer advocate focused on machine learning and generative AI. In 2024, he coauthored the launch of single-document question answering for Amazon Bedrock Knowledge Bases, enabling users to query individual documents without setting up a persistent vector database or ingestion pipeline.

He subsequently led developer relations at Anyscale, working across Ray, distributed training, fine-tuning, and inference optimization, before joining Crusoe.

  • Match retrieval to the document. For scanned forms, illustrated textbooks, and image-heavy instructions, separating text, tables, and images can destroy their shared context. Debnath’s visual document retrieval workflow uses ColPali to embed complete pages as visual patches and Qdrant to identify relevant pages through late-interaction visual retrieval. He recommends this heavier architecture only when simpler text-based retrieval cannot answer the question reliably.
  • Build agents from working tools. Debnath helped introduce Strands Agents, an open-source SDK that pairs capable models with callable tools while supporting Amazon Bedrock, LiteLLM, and Ollama. His implementation examples extend a functioning retrieval pipeline with spoken answers and connect Model Context Protocol servers to Manim for mathematical animations. His practical distinction: when a specific speech behavior must be dependable, configure the tool directly instead of relying on prompts.
  • Evaluate realistic analytical work. He coauthored DSBC, a benchmark for data-science agents, examining ambiguous instructions, data leakage, temperature sensitivity, and tasks that combine multiple analytical operations. The research tests whether costlier agentic execution actually improves results.
  • Engineer for failure. Debnath’s writing on reliable agent systems argues that tool timeouts, degraded retrieval, corrupted context, and cascading latency demand production-ready agent architecture: durable state, observability, dependency isolation, checkpointing, and recoverable execution.

Read the topics behind these talks

2 conference talks

Key ideas

Scroll to read ↓

A file summary that speaks and a Manim animation generator show how Strands delegates tool selection to a model—and what remains in the surrounding application.

  • How much workflow does an agent need?
    0:19 ↗
  • Read a chapter, save a summary, and speak it
    1:49 ↗
  • Give the agent a mathematical animation tool
    5:03 ↗
  • Run the server, then request the video
    8:24 ↗
  • What the short agent example leaves out
    9:59 ↗
  • Mix custom functions with supplied tools
    12:00 ↗
  • Build and contribute an example
    13:33 ↗

Key ideas

Scroll to read ↓

Build visual document retrieval from page embeddings and MaxSim scoring, then connect the retrieved evidence to a multimodal model and a Strands agent with speech tools.

  • What should a multimodal RAG system retrieve?
    0:37 ↗
  • Three ways to separate retrieval from answer generation
    8:02 ↗
  • When extraction separates things that belong together
    14:12 ↗
  • Represent each page with many vectors
    18:43 ↗
  • Make visual and textual representations comparable
    22:12 ↗
  • Encode pages offline and questions at query time
    27:27 ↗
  • Late interaction turns patch matches into a page score
    31:25 ↗
  • Prepare the local model and vector store
    35:28 ↗
  • Ingest the textbook and retrieve trophic-level evidence
    45:09 ↗
  • Give the answer model images, not retrieval vectors
    50:03 ↗
  • Wrap capabilities as tools
    52:28 ↗
  • Add image reading, then speech
    1:02:11 ↗
  • Separate ingestion cost from search design
    1:12:13 ↗
  • Test the visual assumptions on your own documents
    1:16:37 ↗

References