Framework-Free AI Engineering Hands-On: Free Open-Source Colab-Based RAG, Agents, and Evaluation Released

Hands-on materials titled 'AI Engineer Notebooks' designed for AI Engineers and Forward Deployed Engineers (FDEs) have been released. Designed to help users understand fundamental principles by directly implementing agent loops, RAG, and evaluation systems through raw API calls, the materials do not rely on wrappers such as LangChain or LlamaIndex.

Key Features

• Framework Exclusion: Building systems with raw APIs instead of complex wrappers enhances the ability to understand what frameworks actually do and judge their necessity.
• Evaluation (Evals) Centric: Applying the 'measure then tune' principle to all sections fosters the habit of building production-ready systems rather than just demos.
• Free Execution Environment: The entire course can be run without a credit card using Groq's free API (LoRA fine-tuning and self-hosted serving are provided as Colab GPU appendices).
• Real-World Case Studies: Includes three end-to-end case studies under real-world constraints, such as production debugging, pipeline vs. agent cost comparison, and red team benchmarks.

Learning Content

• Model APIs: Prompt basics, structured output, tool calling, streaming, and context caching.
• RAG: Embeddings, hybrid search, reranking, chunking strategies, and diagnosing failure causes.
• Evaluation: Building golden sets, LLM-as-a-judge, and regression testing.
• Agents: Raw API-based agent loops, tool design, guardrails, MCP, and skill patterns.
• Model Adaptation: Trade-off analysis between fine-tuning, RAG, and prompt engineering.

Since it uses an OpenAI-compatible API, patterns can be easily ported to OpenAI and Anthropic, and it is released under the MIT license.