TinyFish's Bigset Turns Text Prompts into Real-Time Datasets

Bigset, launched by TinyFish, is an open-source multi-agent system that takes a natural language sentence as input and collects data from the web in real time to build a structured dataset. When a user describes what they want, the system automatically infers a schema, deploys autonomous agents to investigate web pages, and after result verification and deduplication, delivers a clean table in CSV or XLSX format. Datasets can be set to automatically refresh at intervals ranging from 30 minutes to a week, keeping them up to date without any separate scripting work.

The system is divided into two agent roles. The Orchestrator agent performs a broad search to identify the rows to be included in the dataset and their locations on the web, while the Sub-agent investigates each individual entity to fill in the data. Sub-agents fetch real-time data via TinyFish Search and Fetch, operating within a limit of 6 tool calls for efficiency.

For data reliability, sub-agents do not fabricate values arbitrarily, leaving unconfirmed fields blank. Technically, it supports schema inference via Claude Sonnet 4.6 and agent roles based on Qwen3.7-max, and can be configured through OpenRouter. Self-hosting is possible via Docker, and it is licensed under AGPL-3.0.

TinyFish, headquartered in Palo Alto, raised a $47 million Series A round led by ICONIQ, and counts Google, DoorDash, and Amazon among its customers. Bigset serves as an open-source alternative to existing paid dataset tools, offering the advantage of allowing users to own the entire pipeline themselves without domain restrictions.