Open Source Data Extraction Service

LangChain has released a hosted version of its open source data extraction service. The service uses large language models (LLMs) to automatically extract structured information from unstructured data such as PDF, HTML, and text. It is not intended for production use, but is designed as a reference for developers to start building their own applications.

The service supports features such as custom schema definition, few-shot example learning, switching LLM models, and sharing extractors between users. In the web frontend, users can define and test extraction schemas in natural language, and it can also be integrated into existing LangChain workflows via a LangServe endpoint.

As a real-world use case, the service demonstrates extracting financial data from Uber's Q4 2023 earnings call PDF. It automatically extracts key financial metrics such as revenue and EBITDA, along with the corresponding evidence sentences. It also demonstrates that when the format of an initial extraction result doesn't match the schema specification, adding few-shot examples can improve the LLM so that it extracts data accurately in the intended format.

Developers can use the service through a Python client. They define a schema with Pydantic, create an extractor via an HTTP request, then upload documents to receive results. It can also be accessed through LangServe's RemoteRunnable interface, allowing it to be combined into more complex pipelines such as vector search.