How Cerebras Built Its Internal Knowledge Base

Cerebras built Cerebras Knowledge to unify internal data fragmented across various tools such as Slack, GitHub, Jira, and Google Docs. It adopted an approach of extracting data without changing existing workflows.

The core structure of the system is as follows:

• Data Collection and Storage: Data is extracted directly from each platform and stored together with embeddings, original text summaries, and metadata in a single Postgres table.
• Single Interface: Regardless of the data source—from Slack threads to code repositories—everything is stored in the same embedding table, enabling immediate querying.
• Scalability: The system aims for a simple structure that lets developers easily build custom connectors for new data sources.

The system now processes over 15,000 questions daily, and has established itself as core infrastructure used not only by humans but also by automated tools and AI agents.