Laya Decision Model: Building a Personal Information Judgment Environment 5-6x Faster Than Qwen3-235B via SageMaker Deployment

·2026.11.16 21:43

Key point

Laya significantly reduces latency compared to large LLMs with 30ms-range judgments on a single T4, while cutting costs through fixed instance fees independent of request volume.

1 / 3

Details

  • The fine-tuned Laya achieved 98.5% accuracy and 94.5% recall on the KDPII test, reaching an error level comparable to Jev and large LLMs
  • On ml.g4dn.xlarge (1 T4), it processes one judgment in 35-39ms, making it 5-6x faster than Qwen3-235B (220-300ms) or Claude Haiku 5.5 (1,250ms)
  • The monthly cost is approximately $661 for Laya, a significant reduction compared to Qwen3-235B (approximately $11,800), with data processed within the customer's AWS account
  • Laya cannot generate text or perform masking, requiring combination with Bedrock Guardrails, and caution is needed regarding truncation when input exceeds 1,024 tokens

Training and Deployment Configuration

  • Trained and served the Apache-2.0 open-source Laya (mmBERT-base, 320 million parameters) using the SageMaker pytorch container
  • Training took 3 hours and 27 minutes on ml.g6e.12xlarge (approximately $56), and serving is recommended on 2 ml.g4dn.xlarge instances
  • The FastAPI-based serving app processes batches of up to 32 items and supports legacy drivers with CUDA compatibility packages

Performance and Cost Comparison

  • Zero-shot Laya has low accuracy at 19.6%, but improves to 98.5% accuracy when retrained with synthetic data and KDPII
  • LLMs incur token-based fees where costs increase proportionally with request volume, whereas Laya remains fixed due to instance-time billing
  • LLMs are cost-effective for under 3.3 million requests per month, but a proprietary Decision Model becomes advantageous when exceeding this volume

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.