Building LLM-Based Products

AI21 Labs explains LLM-based product development by dividing it into four stages: preparation, building, deployment, and monitoring. The key point is to sort out data accessibility, security, and stakeholder roles before choosing a model.

In the preparation stage, not only CEO, CTO, PM but also data scientists, data engineers, software engineers, DevOps, designers, legal/compliance, business analysts, and domain experts are brought together to finalize requirements and constraints. If there is sensitive data that cannot be shared with external vendors, alternative approaches must be designed at this stage first.

In the building stage, work proceeds in the following order.

• Language model selection: decided based on cost, performance, and complexity
• User flow and wireframe: defining input formats and expected outputs
• Data Curation: collecting example inputs and outputs for training and evaluation
• Training / Prompt Engineering: customization through additional training or prompt design
• Parameters adjustment and Model Evaluation: verifying response quality
• Pre-processing / Post-processing: input refinement and output correction

In the e-commerce product description generator example, a three-tier prompt structure is proposed on top of a foundation model, separating Brand Guidelines, Product Details, and Instruction. Separating fixed sections from variable sections secures consistency and scalability for mass generation, but final quality still needs continuous human monitoring.

Deployment can be carried out via API, SDK, or cloud servers, and AI21 models can also be used on Amazon SageMaker, Amazon Bedrock, and Google Cloud Platform. After going through QA and testing before launch, the product must continue to be improved through user response, engagement, revenue impact, and ongoing communication with vendors.