AI Agent 'Keenable SELECT' Released for Searching and Processing Web Data with SQL

Keenable SELECT is an AI agent based on an MCP (Model Context Protocol) server that allows searching, filtering, and extracting web data using SQL syntax. Unlike traditional web search that only provides a list of links, requiring agents to read pages and infer information, SELECT combines with DuckDB to enable direct querying of web data as if it were tables.

Core Working Principle

The agent executes DuckDB SELECT statements via the select tool. The server identifies and executes web search and LLM-based operators within the SQL parser, then substitutes the results into standard columns for the final query execution in DuckDB. This approach enables searching over 1,000 pages in a single call, performing precise filtering with WHERE clauses without LLM costs, and extracting fields through small-scale LLM calls per row.

Key Operators

The following semantic operators can be used within SQL:

• WEB_SEARCH: Searches multiple queries simultaneously, removes duplicates, and merges results.
• WEB_FETCH: Fetches specified URLs in Markdown format and returns them as rows.
• SEM_EXTRACT: Extracts specific fields via per-row LLM calls.
• SEM_MATCH: Used as a WHERE condition for semantic-based filtering.
• SEM_SCORE: Calculates embedding-based scores for sorting.

Agent Architecture

Report generation consists of a two-stage agent process. First, the Research Agent repeatedly calls the select tool to collect data, and then the Report Agent creates an HTML report using dataframes in a server-side sandboxed Python session. The Report Agent undergoes a self-correction process by checking rendered screenshots and JavaScript error counts before publishing the final link.