Giedrius Šteimantas

Giedrius Šteimantas led scraping engineering at Oxylabs at World’s Fair 2026, overseeing scraper API teams and infrastructure for gathering public web data. His work applies the practical lessons of web scraping to AI agents: retrieve usable information, detect failed access before asking a model to interpret it, and choose tools that fit each stage of a task. His reconstruction of a shopping agent explains how those choices affect cost, product selection, and the ability to complete a purchase.

From web-data research to scraping engineering

Šteimantas’s use of web data predates his work on agents. His 2014 master’s thesis at Copenhagen Business School examined subscription streaming and film-release strategies, using scraped web information and downloadable IMDb data. It applied data collection to understanding how a new digital distribution channel affected an established industry.

His Tauragė ambassador biography describes entrepreneurship and investment in information technology and wood processing, work with corporations and smaller startups, and a return to Vilnius after eleven years in Denmark. As an honorary ambassador, he helped connect his home region with international partners.

At Oxylabs, Šteimantas was identified as an engineering manager in the company’s 2025 OxyCon coverage and as director of scraping engineering in its September 2026 program. Its World’s Fair 2026 article describes his engineering responsibilities across Scraping API, Web Unblocker, Headless Browser, and Fast Search API, supporting AI retrieval, agent workflows, and training-data collection. These responsibilities connect web-access infrastructure with a recurring application problem: an agent cannot make a useful decision from a page it failed to retrieve.

Reworking a shopping agent

In The Missing Layer in Agentic AI, Šteimantas describes helping a friend whose chatbot recommended clothing and passed shopping requests to an agent. His friend built the original system; Šteimantas advised on and reworked its web-access stages. The original agent used browser automation to search a fixed set of retailers, inspect products, and make purchases. CAPTCHA challenges interrupted access, heavy pages slowed retrieval, and retries made transaction costs difficult to predict. Location-dependent stock and sizing could also make a product appear available during discovery but unavailable at checkout.

Matching web-access tools to each stage

• Discovery: Šteimantas separates the workflow into discovery, product evaluation, user approval, and purchase. Each stage needs different capabilities. For discovery, he uses Fast Search API so the agent can formulate multiple queries and select candidate URLs from compact search results. This expands the search beyond a predetermined retailer list and avoids loading entire storefronts merely to identify promising product pages.
• Content validation: His central lesson for product evaluation is validate content before compression. He initially considered compressing the large retrieval output, then recognized that much of it was unusable. An HTTP 200 response can contain a CAPTCHA or block page rather than a product description. A model may recognize that failure, but sending the page to it still consumes tokens. In his illustrative example, if only three of ten retrieved pages contain valid content, submitting all ten wastes processing on the seven failed retrievals. Smaller output helps only after the system establishes that it retrieved the right material.
• Product inspection: He therefore rebuilds product inspection around Web Scraper API. In his account, blocked requests produce explicit errors that the application can exclude before calling the model. Successful responses can arrive as Markdown rather than raw HTML, and the application can request many pages in parallel. Dynamic pages may still require browser rendering inside the service; the agent does not need to manage a browser for every inspection. Geolocation options help it check prices, stock, and product details in the customer’s relevant market. Šteimantas also highlights successful-result billing as a way to keep failed retrievals from adding direct scraper charges.
• Checkout: After the agent presents its selection and the user approves it, checkout requires interaction with inputs and changing page state. Here Šteimantas retains the original system’s Playwright MCP approach and connects an Oxylabs headless browser. He describes browser-level stealth, an attached residential proxy, and geolocation as the access capabilities supplied by that replacement. Keeping the geographic context consistent between verification and checkout addresses one cause of mismatched availability. The browser can then select the requested size, add the item to the cart, and proceed through the purchase flow. His principle is to reserve browser automation for necessary interactions, while allowing retrieval infrastructure to handle rendering when a page demands it.

Affordable web data and competition

Šteimantas also argues that affordable access to public web data matters for competition. In his discussion of data gatekeeping, he connects restrictions on collection with consequences for investigative journalism, academic research, price comparison, and smaller companies building AI applications. He describes a reciprocal relationship: agents need fresh external information, while AI-assisted collection can reduce the engineering effort required to obtain it. This position gives his emphasis on cost a wider purpose. Avoiding blocked-page processing and unnecessary browser work can make web-dependent applications more practical for teams with limited resources.