Free Web Research
Key point
Built a free web search and scraping workflow using local Qwen3.5 and MCP tools.
Details
Running Qwen3.5 27B Q3_K_M on an RTX 4090 with about 200K token context, the author built and uses an MCP server (webmcp) for local web research.
There are two core tools.
- search_web: search via DDGS or SearXNG
- extract: Playwright + Readability + markdown conversion, then structured extraction via a local LLM
The author built this combination by referencing Firecrawl-style tools, and explains that without using external APIs, search and extraction run entirely locally. This means that, aside from paying for electricity, essentially unlimited AI research becomes possible.
However, early on the results often included hallucinations and inaccurate information. To reduce this, the author added a separate research system prompt in the style of Claude/Opus, and accuracy improved significantly afterward.
The core points of the prompt are as follows.
- Always call get_current_date before the first search
- Start broad with search and don't jump to conclusions right away
- When a person, organization, or claim appears, cross-verify with additional searches
- When there's a quote or a date, check the original source
- Don't trust extraction on a single pass — verify once more with a different source
The author currently rates this approach as usable at roughly Opus 4.6 level in practice, but adds that it sometimes fails to gather sufficiently broad evidence, causing some information to be missed.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.