OpenAI Batch API Asynchronous Processing

OpenAI made it possible to bundle requests that don't need an immediate response and process them asynchronously with the Batch API. Compared to the synchronous API, it offers 50% lower cost, separate and higher rate limits, and completion within 24 hours.

Key use cases include the following.

• Evaluations
• Large-scale data classification
• Embedding content repositories
• Offline video render job queues

Supported endpoints are responses, chat/completions, embeddings, completions, moderations, images/generations, images/edits, and videos. Input is written as a .jsonl file with each line containing an individual request, and results are mapped using custom_id. A single input file can contain only one model.

The workflow proceeds in the order: upload via Files API → create batch with batches.create() → check status with batches.retrieve() → download results via output_file_id. The result file does not guarantee the input order, so results must be reassembled based on custom_id.

Detailed constraints were also specified.

• Maximum 50,000 requests per batch
• Maximum input file size of 200 MB
• Statuses: validating, failed, in_progress, finalizing, completed, expired, cancelling, cancelled
• Output files are automatically deleted 30 days after completion
• /v1/moderations requires input and does not allow stream=true
• Video batches support JSON only, and input_reference only allows file_id or image_url
• Video results are downloadable for 24 hours after completion