BigSet, an AI Agent That Generates Datasets from Natural Language

BigSet is an AI agent tool that researches, extracts, and verifies information on the web based on natural language sentences entered by users, turning it into a structured dataset (CSV, XLSX).

Key Features and How It Works:

• Automatic Schema Inference: It analyzes the input sentence to determine the data's column names, types, and search locations on its own.
• 6-Stage Pipeline: It operates in the order of sentence input $\rightarrow$ schema inference $\rightarrow$ entity discovery $\rightarrow$ parallel research and verification $\rightarrow$ structured table generation $\rightarrow$ periodic automatic updates.
• Continuous Data Updates: The agent re-runs according to a set interval (e.g., daily, weekly) to keep the data up to date.
• Tech Stack: It utilizes TinyFish's Search/Fetch API, Mastra workflows, Vercel AI SDK, OpenRouter (Claude Sonnet), and more.

It is currently an experimental project released under the AGPL-3.0 license, and can be installed and used locally via npm.