Optimizing and Deploying LLMs for Intel Hardware

Covers how to leverage Optimum-Intel and OpenVINO GenAI to efficiently deploy Transformers models in edge and client environments.

OpenVINO™ is a C++-based AI inference solution that, through its GenAI API, makes it easy to integrate LLMs into C++ or Python applications and boost performance.

The core process is as follows:

• Model export: Use the Optimum-Intel OVModelForCausalLM API or CLI to convert the model into OpenVINO IR (Intermediate Representation) format.
• Model optimization: Apply weight-only quantization via the NNCF (Neural Network Compression Framework) for resource-constrained environments.
• Quantization techniques: Supports not only INT8/INT4 quantization but also data-aware methods such as AWQ and GPTQ, optimizing the balance between accuracy and performance.